User detection by a computing device

ABSTRACT

In some embodiments, an electronic device optionally identifies a person&#39;s face, and optionally performs an action in accordance with the identification. In some embodiments, an electronic device optionally determines a gaze location in a user interface, and optionally performs an action in accordance with the determination. In some embodiments, an electronic device optionally designates a user as being present at a sound-playback device in accordance with a determination that sound-detection criteria and verification criteria have been satisfied. In some embodiments, an electronic device optionally determines whether a person is further or closer than a threshold distance from a display device, and optionally provides a first or second user interface for display on the display device in accordance with the determination. In some embodiments, an electronic device optionally modifies the playing of media content in accordance with a determination that one or more presence criteria are not satisfied.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.17/816,314, filed Jul. 29, 2022 and published on Nov. 17, 2022 as U.S.Publication No. 2022-0365595, which is a continuation of U.S.application Ser. No. 17/123,000, filed Dec. 15, 2020 and published onApr. 8, 2021 as U.S. Publication No. 2021-0103333, which is acontinuation of U.S. application Ser. No. 16/881,599, filed May 22, 2020and issued on Apr. 19, 2022 as U.S. Pat. No. 11,307,657, which is acontinuation of U.S. application Ser. No. 15/644,639, filed on Jul. 7,2017 and issued on May 26, 2020 as U.S. Pat. No. 10,664,048, which is acontinuation of U.S. application Ser. No. 14/531,874, filed on Nov. 3,2014 and issued on Sep. 19, 2017 as U.S. Pat. No. 9,766,702, whichclaims priority to U.S. Provisional Application No. 62/014,541, filed onJun. 19, 2014, which are hereby incorporated herein by reference intheir entireties for all purposes.

FIELD OF THE DISCLOSURE

This relates generally to electronic devices that are capable ofdetecting one or more attributes relating to user presence at a device.

BACKGROUND OF THE DISCLOSURE

User interaction with devices such as computers and other electroniccomputing devices has increased significantly in recent years. Thesedevices can be devices such as computers, tablets, televisions,multimedia devices, and the like.

Interaction with such devices can be performed using various inputdevices, such as touch screen displays, touch-sensitive surfaces, remotecontrols, mice and other input devices. However, there are instanceswhere user interaction with the computing devices would be enhanced ifthe user were not required to physically provide input to the computingdevices, but rather if the computing devices were to take certainactions autonomously based on user detection. This latter considerationis particularly important where input devices are battery-operated.

SUMMARY OF THE DISCLOSURE

The embodiments described in this disclosure are directed to one or moredevices that optionally perform facial identification, gazedetermination, audio-based presence detection, distance determination,and presence determination with respect to one or more users of thedevices, and one or more actions that the devices optionally perform inresponse to the above. The full descriptions of the embodiments areprovided in the Drawings and the Detailed Description, and it isunderstood that the Summary provided above does not limit the scope ofthe disclosure in any way.

BRIEF DESCRIPTION OF THE DRAWINGS

For a better understanding of the various described embodiments,reference should be made to the Detailed Description below, inconjunction with the following drawings in which like reference numeralsrefer to corresponding parts throughout the figures.

FIG. 1A is a block diagram illustrating a multifunction device with atouch-sensitive display in accordance with some embodiments.

FIG. 1B is a block diagram illustrating exemplary components for eventhandling in accordance with some embodiments.

FIG. 2 illustrates a multifunction device having a touch screen inaccordance with some embodiments.

FIG. 3 is a block diagram of an exemplary multifunction device with adisplay and a touch-sensitive surface in accordance with someembodiments.

FIG. 4 illustrates an exemplary user interface for a multifunctiondevice with a touch-sensitive surface that is separate from the displayin accordance with some embodiments.

FIG. 5 illustrates a block diagram of an exemplary architecture for thedevice according to some embodiments of the disclosure.

FIGS. 6A-6D illustrate exemplary ways in which interactions withelectronic devices are enhanced based on facial identification inaccordance with some embodiments of the disclosure.

FIGS. 7A-7E are flow diagrams illustrating a method of performing anaction based in part on identification of a face in an image inaccordance with some embodiments.

FIGS. 8A-8E illustrate exemplary ways in which interactions with userinterfaces are enhanced based on gaze determination in accordance withsome embodiments of the disclosure.

FIGS. 9A-9C are flow diagrams illustrating a method of performing anaction based in part on determining a gaze location in accordance withsome embodiments.

FIG. 10 illustrates exemplary ways in which a device can determinewhether a person is present at the device in accordance with someembodiments of the disclosure.

FIGS. 11A-11B are flow diagrams illustrating a method of performingpresence detection in accordance with some embodiments.

FIGS. 12A-12B illustrate exemplary ways in which different userinterfaces can be presented at different distances in accordance withsome embodiments of the disclosure.

FIGS. 13A-13D are flow diagrams illustrating a method of providingdifferent user interfaces at different distances in accordance with someembodiments.

FIGS. 14A-14B illustrate exemplary ways in which a person's experiencewith a device can be enhanced based on presence detection in accordancewith some embodiments of the disclosure.

FIGS. 15A-15C are flow diagrams illustrating a method of modifying mediacontent playback based on presence detection in accordance with someembodiments.

FIGS. 16-20 are functional block diagrams of electronic devices inaccordance with some embodiments.

DETAILED DESCRIPTION

In the following description of embodiments, reference is made to theaccompanying drawings which form a part hereof, and in which it is shownby way of illustration specific embodiments that are optionallypracticed. It is to be understood that other embodiments are optionallyused and structural changes are optionally made without departing fromthe scope of the disclosed embodiments.

Exemplary Devices

Embodiments of electronic devices, user interfaces for such devices, andassociated processes for using such devices are described. In someembodiments, the device is a portable communications device, such as amobile telephone, that also contains other functions, such as PDA and/ormusic player functions. Exemplary embodiments of portable multifunctiondevices include, without limitation, the iPhone®, iPod Touch®, and iPad®devices from Apple Inc. of Cupertino, Calif. Other portable electronicdevices, such as laptops or tablet computers with touch-sensitivesurfaces (e.g., touch screen displays and/or touch pads), are,optionally, used. It should also be understood that, in someembodiments, the device is not a portable communications device, but isa desktop computer or a television with a touch-sensitive surface (e.g.,a touch screen display and/or a touch pad). In some embodiments, thedevice does not have a touch screen display and/or a touch pad, butrather is capable of outputting display information (such as the userinterfaces of the disclosure) for display on a separate display device,and capable of receiving input information from a separate input devicehaving one or more input mechanisms (such as one or more buttons, atouch screen display and/or a touch pad). In some embodiments, thedevice has a display, but is capable of receiving input information froma separate input device having one or more input mechanisms (such as oneor more buttons, a touch screen display and/or a touch pad).

In the discussion that follows, an electronic device that includes adisplay and a touch-sensitive surface is described. It should beunderstood, however, that the electronic device optionally includes oneor more other physical user-interface devices, such as a physicalkeyboard, a mouse and/or a joystick. Further, as described above, itshould be understood that the described electronic device, display andtouch-sensitive surface are optionally distributed amongst two or moredevices. Therefore, as used in this disclosure, information displayed onthe electronic device or by the electronic device is optionally used todescribe information outputted by the electronic device for display on aseparate display device (touch-sensitive or not). Similarly, as used inthis disclosure, input received on the electronic device (e.g., touchinput received on a touch-sensitive surface of the electronic device) isoptionally used to describe input received on a separate input device,from which the electronic device receives input information.

The device typically supports a variety of applications, such as one ormore of the following: a drawing application, a presentationapplication, a word processing application, a website creationapplication, a disk authoring application, a spreadsheet application, agaming application, a telephone application, a video conferencingapplication, an e-mail application, an instant messaging application, aworkout support application, a photo management application, a digitalcamera application, a digital video camera application, a web browsingapplication, a digital music player application, a television channelbrowsing application, and/or a digital video player application.

The various applications that are executed on the device optionally useat least one common physical user-interface device, such as thetouch-sensitive surface. One or more functions of the touch-sensitivesurface as well as corresponding information displayed on the deviceare, optionally, adjusted and/or varied from one application to the nextand/or within a respective application. In this way, a common physicalarchitecture (such as the touch-sensitive surface) of the deviceoptionally supports the variety of applications with user interfacesthat are intuitive and transparent to the user.

Attention is now directed toward embodiments of portable or non-portabledevices with touch-sensitive displays, though the devices need notinclude touch-sensitive displays or displays in general, as describedabove. FIG. 1A is a block diagram illustrating portable or non-portablemultifunction device 100 with touch-sensitive displays 112 in accordancewith some embodiments. Touch-sensitive display 112 is sometimes called a“touch screen” for convenience, and is sometimes known as or called atouch-sensitive display system. Device 100 includes memory 102 (whichoptionally includes one or more computer readable storage mediums),memory controller 122, one or more processing units (CPU's) 120,peripherals interface 118, RF circuitry 108, audio circuitry 110,speaker 111, microphone 113, input/output (I/O) subsystem 106, otherinput or control devices 116, and external port 124. Device 100optionally includes one or more optical sensors 164. Device 100optionally includes one or more intensity sensors 165 for detectingintensity of contacts on device 100 (e.g., a touch-sensitive surfacesuch as touch-sensitive display system 112 of device 100). Device 100optionally includes one or more tactile output generators 167 forgenerating tactile outputs on device 100 (e.g., generating tactileoutputs on a touch-sensitive surface such as touch-sensitive displaysystem 112 of device 100 or touchpad 355 of device 300). Thesecomponents optionally communicate over one or more communication busesor signal lines 103.

As used in the specification and claims, the term “intensity” of acontact on a touch-sensitive surface refers to the force or pressure(force per unit area) of a contact (e.g., a finger contact) on thetouch-sensitive surface, or to a substitute (proxy) for the force orpressure of a contact on the touch-sensitive surface. The intensity of acontact has a range of values that includes at least four distinctvalues and more typically includes hundreds of distinct values (e.g., atleast 256). Intensity of a contact is, optionally, determined (ormeasured) using various approaches and various sensors or combinationsof sensors. For example, one or more force sensors underneath oradjacent to the touch-sensitive surface are, optionally, used to measureforce at various points on the touch-sensitive surface. In someimplementations, force measurements from multiple force sensors arecombined (e.g., a weighted average) to determine an estimated force of acontact. Similarly, a pressure-sensitive tip of a stylus is, optionally,used to determine a pressure of the stylus on the touch-sensitivesurface. Alternatively, the size of the contact area detected on thetouch-sensitive surface and/or changes thereto, the capacitance of thetouch-sensitive surface proximate to the contact and/or changes thereto,and/or the resistance of the touch-sensitive surface proximate to thecontact and/or changes thereto are, optionally, used as a substitute forthe force or pressure of the contact on the touch-sensitive surface. Insome implementations, the substitute measurements for contact force orpressure are used directly to determine whether an intensity thresholdhas been exceeded (e.g., the intensity threshold is described in unitscorresponding to the substitute measurements). In some implementations,the substitute measurements for contact force or pressure are convertedto an estimated force or pressure and the estimated force or pressure isused to determine whether an intensity threshold has been exceeded(e.g., the intensity threshold is a pressure threshold measured in unitsof pressure).

As used in the specification and claims, the term “tactile output”refers to physical displacement of a device relative to a previousposition of the device, physical displacement of a component (e.g., atouch-sensitive surface) of a device relative to another component(e.g., housing) of the device, or displacement of the component relativeto a center of mass of the device that will be detected by a user withthe user's sense of touch. For example, in situations where the deviceor the component of the device is in contact with a surface of a userthat is sensitive to touch (e.g., a finger, palm, or other part of auser's hand), the tactile output generated by the physical displacementwill be interpreted by the user as a tactile sensation corresponding toa perceived change in physical characteristics of the device or thecomponent of the device. For example, movement of a touch-sensitivesurface (e.g., a touch-sensitive display or trackpad) is, optionally,interpreted by the user as a “down click” or “up click” of a physicalactuator button. In some cases, a user will feel a tactile sensationsuch as an “down click” or “up click” even when there is no movement ofa physical actuator button associated with the touch-sensitive surfacethat is physically pressed (e.g., displaced) by the user's movements. Asanother example, movement of the touch-sensitive surface is, optionally,interpreted or sensed by the user as “roughness” of the touch-sensitivesurface, even when there is no change in smoothness of thetouch-sensitive surface. While such interpretations of touch by a userwill be subject to the individualized sensory perceptions of the user,there are many sensory perceptions of touch that are common to a largemajority of users. Thus, when a tactile output is described ascorresponding to a particular sensory perception of a user (e.g., an “upclick,” a “down click,” “roughness”), unless otherwise stated, thegenerated tactile output corresponds to physical displacement of thedevice or a component thereof that will generate the described sensoryperception for a typical (or average) user.

It should be appreciated that device 100 is only one example of aportable or non-portable multifunction device, and that device 100optionally has more or fewer components than shown, optionally combinestwo or more components, or optionally has a different configuration orarrangement of the components. The various components shown in FIG. 1Aare implemented in hardware, software, or a combination of both hardwareand software, including one or more signal processing and/or applicationspecific integrated circuits. Further, the various components shown inFIG. 1A are optionally implemented across two or more devices; forexample, a display and audio circuitry on a display device, atouch-sensitive surface on an input device, and remaining components ondevice 100. In such an embodiment, device 100 optionally communicateswith the display device and/or the input device to facilitate operationof the system, as described in the disclosure, and the variouscomponents described herein that relate to display and/or input remainin device 100, or are optionally included in the display and/or inputdevice, as appropriate.

Memory 102 optionally includes high-speed random access memory andoptionally also includes non-volatile memory, such as one or moremagnetic disk storage devices, flash memory devices, or othernon-volatile solid-state memory devices. Access to memory 102 by othercomponents of device 100, such as CPU 120 and the peripherals interface118, is, optionally, controlled by memory controller 122.

Peripherals interface 118 can be used to couple input and outputperipherals of the device to CPU 120 and memory 102. The one or moreprocessors 120 run or execute various software programs and/or sets ofinstructions stored in memory 102 to perform various functions fordevice 100 and to process data.

In some embodiments, peripherals interface 118, CPU 120, and memorycontroller 122 are, optionally, implemented on a single chip, such aschip 104. In some other embodiments, they are, optionally, implementedon separate chips.

RF (radio frequency) circuitry 108 receives and sends RF signals, alsocalled electromagnetic signals. RF circuitry 108 converts electricalsignals to/from electromagnetic signals and communicates withcommunications networks and other communications devices via theelectromagnetic signals. RF circuitry 108 optionally includes well-knowncircuitry for performing these functions, including but not limited toan antenna system, an RF transceiver, one or more amplifiers, a tuner,one or more oscillators, a digital signal processor, a CODEC chipset, asubscriber identity module (SIM) card, memory, and so forth. RFcircuitry 108 optionally communicates with networks, such as theInternet, also referred to as the World Wide Web (WWW), an intranetand/or a wireless network, such as a cellular telephone network, awireless local area network (LAN) and/or a metropolitan area network(MAN), and other devices by wireless communication. The wirelesscommunication optionally uses any of a plurality of communicationsstandards, protocols and technologies, including but not limited toGlobal System for Mobile Communications (GSM), Enhanced Data GSMEnvironment (EDGE), high-speed downlink packet access (HSDPA),high-speed uplink packet access (HSDPA), Evolution, Data-Only (EV-DO),HSPA, HSPA+, Dual-Cell HSPA (DC-HSPDA), long term evolution (LTE), nearfield communication (NFC), wideband code division multiple access(W-CDMA), code division multiple access (CDMA), time division multipleaccess (TDMA), Bluetooth, Wireless Fidelity (Wi-Fi) (e.g., IEEE 802.11a,IEEE 802.11b, IEEE 802.11g and/or IEEE 802.11n), voice over InternetProtocol (VoIP), Wi-MAX, a protocol for e-mail (e.g., Internet messageaccess protocol (IMAP) and/or post office protocol (POP)), instantmessaging (e.g., extensible messaging and presence protocol (XMPP),Session Initiation Protocol for Instant Messaging and PresenceLeveraging Extensions (SIMPLE), Instant Messaging and Presence Service(IMPS)), and/or Short Message Service (SMS), or any other suitablecommunication protocol, including communication protocols not yetdeveloped as of the filing date of this document.

Audio circuitry 110, speaker 111, and microphone 113 provide an audiointerface between a user and device 100. Audio circuitry 110 receivesaudio data from peripherals interface 118, converts the audio data to anelectrical signal, and transmits the electrical signal to speaker 111.Speaker 111 converts the electrical signal to human-audible sound waves.Audio circuitry 110 also receives electrical signals converted bymicrophone 113 from sound waves. Audio circuitry 110 converts theelectrical signal to audio data and transmits the audio data toperipherals interface 118 for processing. Audio data is, optionally,retrieved from and/or transmitted to memory 102 and/or RF circuitry 108by peripherals interface 118. In some embodiments, audio circuitry 110also includes a headset jack (e.g., 212, FIG. 2 ). The headset jackprovides an interface between audio circuitry 110 and removable audioinput/output peripherals, such as output-only headphones or a headsetwith both output (e.g., a headphone for one or both ears) and input(e.g., a microphone).

I/O subsystem 106 couples input/output peripherals on device 100, suchas touch screen 112 and other input control devices 116, to peripheralsinterface 118. I/O subsystem 106 optionally includes display controller156, optical sensor controller 158, intensity sensor controller 159,haptic feedback controller 161 and one or more input controllers 160 forother input or control devices. The one or more input controllers 160receive/send electrical signals from/to other input or control devices116. The other input control devices 116 optionally include physicalbuttons (e.g., push buttons, rocker buttons, etc.), dials, sliderswitches, joysticks, click wheels, and so forth. In some alternateembodiments, input controller(s) 160 are, optionally, coupled to any (ornone) of the following: a keyboard, infrared port, USB port, and apointer device such as a mouse. The one or more buttons (e.g., 208, FIG.2 ) optionally include an up/down button for volume control of speaker111 and/or microphone 113. The one or more buttons optionally include apush button (e.g., 206, FIG. 2 ).

Touch-sensitive display 112 provides an input interface and an outputinterface between the device and a user. As described above, thetouch-sensitive operation and the display operation of touch-sensitivedisplay 112 are optionally separated from each other, such that adisplay device is used for display purposes and a touch-sensitivesurface (whether display or not) is used for input detection purposes,and the described components and functions are modified accordingly.However, for simplicity, the following description is provided withreference to a touch-sensitive display. Display controller 156 receivesand/or sends electrical signals from/to touch screen 112. Touch screen112 displays visual output to the user. The visual output optionallyincludes graphics, text, icons, video, and any combination thereof(collectively termed “graphics”). In some embodiments, some or all ofthe visual output corresponds to user-interface objects.

Touch screen 112 has a touch-sensitive surface, sensor or set of sensorsthat accepts input from the user based on haptic and/or tactile contact.Touch screen 112 and display controller 156 (along with any associatedmodules and/or sets of instructions in memory 102) detect contact (andany movement or breaking of the contact) on touch screen 112 andconverts the detected contact into interaction with user-interfaceobjects (e.g., one or more soft keys, icons, web pages or images) thatare displayed on touch screen 112. In an exemplary embodiment, a pointof contact between touch screen 112 and the user corresponds to a fingerof the user.

Touch screen 112 optionally uses LCD (liquid crystal display)technology, LPD (light emitting polymer display) technology, or LED(light emitting diode) technology, although other display technologiesare used in other embodiments. Touch screen 112 and display controller156 optionally detect contact and any movement or breaking thereof usingany of a plurality of touch sensing technologies now known or laterdeveloped, including but not limited to capacitive, resistive, infrared,and surface acoustic wave technologies, as well as other proximitysensor arrays or other elements for determining one or more points ofcontact with touch screen 112. In an exemplary embodiment, projectedmutual capacitance sensing technology is used, such as that found in theiPhone®, iPod Touch®, and iPad® from Apple Inc. of Cupertino, Calif.

Touch screen 112 optionally has a video resolution in excess of 100 dpi.In some embodiments, the touch screen has a video resolution ofapproximately 160 dpi. The user optionally makes contact with touchscreen 112 using any suitable object or appendage, such as a stylus, afinger, and so forth. In some embodiments, the user interface isdesigned to work primarily with finger-based contacts and gestures,which can be less precise than stylus-based input due to the larger areaof contact of a finger on the touch screen. In some embodiments, thedevice translates the rough finger-based input into a precisepointer/cursor position or command for performing the actions desired bythe user.

In some embodiments, in addition to the touch screen, device 100optionally includes a touchpad (not shown) for activating ordeactivating particular functions. In some embodiments, the touchpad isa touch-sensitive area of the device that, unlike the touch screen, doesnot display visual output. The touchpad is, optionally, atouch-sensitive surface that is separate from touch screen 112 or anextension of the touch-sensitive surface formed by the touch screen.

Device 100 also includes power system 162 for powering the variouscomponents. Power system 162 optionally includes a power managementsystem, one or more power sources (e.g., battery, alternating current(AC)), a recharging system, a power failure detection circuit, a powerconverter or inverter, a power status indicator (e.g., a light-emittingdiode (LED)) and any other components associated with the generation,management and distribution of power in portable or non-portabledevices.

Device 100 optionally also includes one or more optical sensors 164.FIG. 1A shows an optical sensor coupled to optical sensor controller 158in I/O subsystem 106. Optical sensor 164 optionally includescharge-coupled device (CCD) or complementary metal-oxide semiconductor(CMOS) phototransistors. Optical sensor 164 receives light from theenvironment, projected through one or more lens, and converts the lightto data representing an image. In conjunction with imaging module 143(also called a camera module), optical sensor 164 optionally capturesstill images or video. In some embodiments, an optical sensor is locatedon the back of device 100, opposite touch screen display 112 on thefront of the device, so that the touch screen display is enabled for useas a viewfinder for still and/or video image acquisition. In someembodiments, another optical sensor is located on the front of thedevice so that the user's image is, optionally, obtained forvideoconferencing while the user views the other video conferenceparticipants on the touch screen display.

Device 100 optionally also includes one or more contact intensitysensors 165. FIG. 1A shows a contact intensity sensor coupled tointensity sensor controller 159 in I/O subsystem 106. Contact intensitysensor 165 optionally includes one or more piezoresistive strain gauges,capacitive force sensors, electric force sensors, piezoelectric forcesensors, optical force sensors, capacitive touch-sensitive surfaces, orother intensity sensors (e.g., sensors used to measure the force (orpressure) of a contact on a touch-sensitive surface). Contact intensitysensor 165 receives contact intensity information (e.g., pressureinformation or a proxy for pressure information) from the environment.In some embodiments, at least one contact intensity sensor is collocatedwith, or proximate to, a touch-sensitive surface (e.g., touch-sensitivedisplay system 112). In some embodiments, at least one contact intensitysensor is located on the back of device 100, opposite touch screendisplay 112 which is located on the front of device 100.

Device 100 optionally also includes one or more proximity sensors 166.FIG. 1A shows proximity sensor 166 coupled to peripherals interface 118.Alternately, proximity sensor 166 is coupled to input controller 160 inI/O subsystem 106. In some embodiments, the proximity sensor turns offand disables touch screen 112 when the multifunction device is placednear the user's ear (e.g., when the user is making a phone call).

Device 100 optionally also includes one or more tactile outputgenerators 167. FIG. 1A shows a tactile output generator coupled tohaptic feedback controller 161 in I/O subsystem 106. Tactile outputgenerator 167 optionally includes one or more electroacoustic devicessuch as speakers or other audio components and/or electromechanicaldevices that convert energy into linear motion such as a motor,solenoid, electroactive polymer, piezoelectric actuator, electrostaticactuator, or other tactile output generating component (e.g., acomponent that converts electrical signals into tactile outputs on thedevice). Contact intensity sensor 165 receives tactile feedbackgeneration instructions from haptic feedback module 133 and generatestactile outputs on device 100 that are capable of being sensed by a userof device 100. In some embodiments, at least one tactile outputgenerator is collocated with, or proximate to, a touch-sensitive surface(e.g., touch-sensitive display system 112) and, optionally, generates atactile output by moving the touch-sensitive surface vertically (e.g.,in/out of a surface of device 100) or laterally (e.g., back and forth inthe same plane as a surface of device 100). In some embodiments, atleast one tactile output generator sensor is located on the back ofdevice 100, opposite touch screen display 112 which is located on thefront of device 100.

Device 100 optionally also includes one or more accelerometers 168. FIG.1A shows accelerometer 168 coupled to peripherals interface 118.Alternately, accelerometer 168 is, optionally, coupled to an inputcontroller 160 in I/O subsystem 106. In some embodiments, information isdisplayed on the touch screen display in a portrait view or a landscapeview based on an analysis of data received from the one or moreaccelerometers. Device 100 optionally includes, in addition toaccelerometer(s) 168, a magnetometer (not shown) and a GPS (or GLONASSor other global navigation system) receiver (not shown) for obtaininginformation concerning the location and orientation (e.g., portrait orlandscape) of device 100.

In some embodiments, the software components stored in memory 102include operating system 126, communication module (or set ofinstructions) 128, contact/motion module (or set of instructions) 130,graphics module (or set of instructions) 132, text input module (or setof instructions) 134, Global Positioning System (GPS) module (or set ofinstructions) 135, and applications (or sets of instructions) 136.Furthermore, in some embodiments memory 102 stores device/globalinternal state 157, as shown in FIGS. 1A and 3 . Device/global internalstate 157 includes one or more of: active application state, indicatingwhich applications, if any, are currently active; display state,indicating what applications, views or other information occupy variousregions of touch screen display 112; sensor state, including informationobtained from the device's various sensors and input control devices116; and location information concerning the device's location and/orattitude.

Operating system 126 (e.g., Darwin, RTXC, LINUX, UNIX, OS X, WINDOWS, oran embedded operating system such as VxWorks) includes various softwarecomponents and/or drivers for controlling and managing general systemtasks (e.g., memory management, storage device control, powermanagement, etc.) and facilitates communication between various hardwareand software components.

Communication module 128 facilitates communication with other devicesover one or more external ports 124 and also includes various softwarecomponents for handling data received by RF circuitry 108 and/orexternal port 124. External port 124 (e.g., Universal Serial Bus (USB),FIREWIRE, etc.) is adapted for coupling directly to other devices orindirectly over a network (e.g., the Internet, wireless LAN, etc.). Insome embodiments, the external port is a multi-pin (e.g., 30-pin)connector that is the same as, or similar to and/or compatible with the30-pin connector used on iPod (trademark of Apple Inc.) devices.

Contact/motion module 130 optionally detects contact with touch screen112 (in conjunction with display controller 156) and othertouch-sensitive devices (e.g., a touchpad or physical click wheel).Contact/motion module 130 includes various software components forperforming various operations related to detection of contact, such asdetermining if contact has occurred (e.g., detecting a finger-downevent), determining an intensity of the contact (e.g., the force orpressure of the contact or a substitute for the force or pressure of thecontact) determining if there is movement of the contact and trackingthe movement across the touch-sensitive surface (e.g., detecting one ormore finger-dragging events), and determining if the contact has ceased(e.g., detecting a finger-up event or a break in contact).Contact/motion module 130 receives contact data from the touch-sensitivesurface. Determining movement of the point of contact, which isrepresented by a series of contact data, optionally includes determiningspeed (magnitude), velocity (magnitude and direction), and/or anacceleration (a change in magnitude and/or direction) of the point ofcontact. These operations are, optionally, applied to single contacts(e.g., one finger contacts) or to multiple simultaneous contacts (e.g.,“multitouch”/multiple finger contacts). In some embodiments,contact/motion module 130 and display controller 156 detect contact on atouchpad.

In some embodiments, contact/motion module 130 uses a set of one or moreintensity thresholds to determine whether an operation has beenperformed by a user (e.g., to determine whether a user has “clicked” onan icon). In some embodiments at least a subset of the intensitythresholds are determined in accordance with software parameters (e.g.,the intensity thresholds are not determined by the activation thresholdsof particular physical actuators and can be adjusted without changingthe physical hardware of device 100). For example, a mouse “click”threshold of a trackpad or touch screen display can be set to any of alarge range of predefined thresholds values without changing thetrackpad or touch screen display hardware. Additionally, in someimplementations a user of the device is provided with software settingsfor adjusting one or more of the set of intensity thresholds (e.g., byadjusting individual intensity thresholds and/or by adjusting aplurality of intensity thresholds at once with a system-level click“intensity” parameter).

Contact/motion module 130 optionally detects a gesture input by a user.Different gestures on the touch-sensitive surface have different contactpatterns and intensities. Thus, a gesture is, optionally, detected bydetecting a particular contact pattern. For example, detecting a fingertap gesture includes detecting a finger-down event followed by detectinga finger-up (lift off) event at the same position (or substantially thesame position) as the finger-down event (e.g., at the position of anicon). As another example, detecting a finger swipe gesture on thetouch-sensitive surface includes detecting a finger-down event followedby detecting one or more finger-dragging events, and subsequentlyfollowed by detecting a finger-up (lift off) event.

Graphics module 132 includes various known software components forrendering and displaying graphics on touch screen 112 or other display,including components for changing the visual impact (e.g., brightness,transparency, saturation, contrast or other visual property) of graphicsthat are displayed. As used herein, the term “graphics” includes anyobject that can be displayed to a user, including without limitationtext, web pages, icons (such as user-interface objects including softkeys), digital images, videos, animations and the like.

In some embodiments, graphics module 132 stores data representinggraphics to be used. Each graphic is, optionally, assigned acorresponding code. Graphics module 132 receives, from applicationsetc., one or more codes specifying graphics to be displayed along with,if necessary, coordinate data and other graphic property data, and thengenerates screen image data to output to display controller 156.

Haptic feedback module 133 includes various software components forgenerating instructions used by tactile output generator(s) 167 toproduce tactile outputs at one or more locations on device 100 inresponse to user interactions with device 100.

Text input module 134, which is, optionally, a component of graphicsmodule 132, provides soft keyboards for entering text in variousapplications (e.g., contacts 137, e-mail 140, IM 141, browser 147, andany other application that needs text input).

GPS module 135 determines the location of the device and provides thisinformation for use in various applications (e.g., to telephone 138 foruse in location-based dialing, to camera 143 as picture/video metadata,and to applications that provide location-based services such as weatherwidgets, local yellow page widgets, and map/navigation widgets).

Applications 136 optionally include the following modules (or sets ofinstructions), or a subset or superset thereof:

-   -   contacts module 137 (sometimes called an address book or contact        list);    -   telephone module 138;    -   video conferencing module 139;    -   e-mail client module 140;    -   instant messaging (IM) module 141;    -   workout support module 142;    -   camera module 143 for still and/or video images;    -   image management module 144;    -   browser module 147;    -   calendar module 148;    -   widget modules 149, which optionally include one or more of:        weather widget 149-1, stocks widget 149-2, calculator widget        149-3, alarm clock widget 149-4, dictionary widget 149-5, and        other widgets obtained by the user, as well as user-created        widgets 149-6;    -   widget creator module 150 for making user-created widgets 149-6;    -   search module 151;    -   video and music player module 152, which is, optionally, made up        of a video player module and a music player module;    -   notes module 153;    -   map module 154;    -   online video module 155.

Examples of other applications 136 that are, optionally, stored inmemory 102 include other word processing applications, other imageediting applications, drawing applications, presentation applications,JAVA-enabled applications, encryption, digital rights management, voicerecognition, and voice replication.

In conjunction with touch screen 112, display controller 156, contactmodule 130, graphics module 132, and text input module 134, contactsmodule 137 are, optionally, used to manage an address book or contactlist (e.g., stored in application internal state 192 of contacts module137 in memory 102 or memory 370), including: adding name(s) to theaddress book; deleting name(s) from the address book; associatingtelephone number(s), e-mail address(es), physical address(es) or otherinformation with a name; associating an image with a name; categorizingand sorting names; providing telephone numbers or e-mail addresses toinitiate and/or facilitate communications by telephone 138, videoconference 139, e-mail 140, or IM 141; and so forth.

In conjunction with RF circuitry 108, audio circuitry 110, speaker 111,microphone 113, touch screen 112, display controller 156, contact module130, graphics module 132, and text input module 134, telephone module138 are, optionally, used to enter a sequence of characterscorresponding to a telephone number, access one or more telephonenumbers in address book 137, modify a telephone number that has beenentered, dial a respective telephone number, conduct a conversation anddisconnect or hang up when the conversation is completed. As notedabove, the wireless communication optionally uses any of a plurality ofcommunications standards, protocols and technologies.

In conjunction with RF circuitry 108, audio circuitry 110, speaker 111,microphone 113, touch screen 112, display controller 156, optical sensor164, optical sensor controller 158, contact module 130, graphics module132, text input module 134, contact list 137, and telephone module 138,videoconferencing module 139 includes executable instructions toinitiate, conduct, and terminate a video conference between a user andone or more other participants in accordance with user instructions.

In conjunction with RF circuitry 108, touch screen 112, displaycontroller 156, contact module 130, graphics module 132, and text inputmodule 134, e-mail client module 140 includes executable instructions tocreate, send, receive, and manage e-mail in response to userinstructions. In conjunction with image management module 144, e-mailclient module 140 makes it very easy to create and send e-mails withstill or video images taken with camera module 143.

In conjunction with RF circuitry 108, touch screen 112, displaycontroller 156, contact module 130, graphics module 132, and text inputmodule 134, the instant messaging module 141 includes executableinstructions to enter a sequence of characters corresponding to aninstant message, to modify previously entered characters, to transmit arespective instant message (for example, using a Short Message Service(SMS) or Multimedia Message Service (MMS) protocol for telephony-basedinstant messages or using XMPP, SIMPLE, or IMPS for Internet-basedinstant messages), to receive instant messages and to view receivedinstant messages. In some embodiments, transmitted and/or receivedinstant messages optionally include graphics, photos, audio files, videofiles and/or other attachments as are supported in a MMS and/or anEnhanced Messaging Service (EMS). As used herein, “instant messaging”refers to both telephony-based messages (e.g., messages sent using SMSor MMS) and Internet-based messages (e.g., messages sent using XMPP,SIMPLE, or IMPS).

In conjunction with RF circuitry 108, touch screen 112, displaycontroller 156, contact module 130, graphics module 132, text inputmodule 134, GPS module 135, map module 154, and music player module 146,workout support module 142 includes executable instructions to createworkouts (e.g., with time, distance, and/or calorie burning goals);communicate with workout sensors (sports devices); receive workoutsensor data; calibrate sensors used to monitor a workout; select andplay music for a workout; and display, store and transmit workout data.

In conjunction with touch screen 112, display controller 156, opticalsensor(s) 164, optical sensor controller 158, contact module 130,graphics module 132, and image management module 144, camera module 143includes executable instructions to capture still images or video(including a video stream) and store them into memory 102, modifycharacteristics of a still image or video, or delete a still image orvideo from memory 102.

In conjunction with touch screen 112, display controller 156, contactmodule 130, graphics module 132, text input module 134, and cameramodule 143, image management module 144 includes executable instructionsto arrange, modify (e.g., edit), or otherwise manipulate, label, delete,present (e.g., in a digital slide show or album), and store still and/orvideo images.

In conjunction with RF circuitry 108, touch screen 112, display systemcontroller 156, contact module 130, graphics module 132, and text inputmodule 134, browser module 147 includes executable instructions tobrowse the Internet in accordance with user instructions, includingsearching, linking to, receiving, and displaying web pages or portionsthereof, as well as attachments and other files linked to web pages.

In conjunction with RF circuitry 108, touch screen 112, display systemcontroller 156, contact module 130, graphics module 132, text inputmodule 134, e-mail client module 140, and browser module 147, calendarmodule 148 includes executable instructions to create, display, modify,and store calendars and data associated with calendars (e.g., calendarentries, to do lists, etc.) in accordance with user instructions.

In conjunction with RF circuitry 108, touch screen 112, display systemcontroller 156, contact module 130, graphics module 132, text inputmodule 134, and browser module 147, widget modules 149 aremini-applications that are, optionally, downloaded and used by a user(e.g., weather widget 149-1, stocks widget 149-2, calculator widget149-3, alarm clock widget 149-4, and dictionary widget 149-5) or createdby the user (e.g., user-created widget 149-6). In some embodiments, awidget includes an HTML (Hypertext Markup Language) file, a CSS(Cascading Style Sheets) file, and a JavaScript file. In someembodiments, a widget includes an XML (Extensible Markup Language) fileand a JavaScript file (e.g., Yahoo! Widgets).

In conjunction with RF circuitry 108, touch screen 112, display systemcontroller 156, contact module 130, graphics module 132, text inputmodule 134, and browser module 147, the widget creator module 150 are,optionally, used by a user to create widgets (e.g., turning auser-specified portion of a web page into a widget).

In conjunction with touch screen 112, display system controller 156,contact module 130, graphics module 132, and text input module 134,search module 151 includes executable instructions to search for text,music, sound, image, video, and/or other files in memory 102 that matchone or more search criteria (e.g., one or more user-specified searchterms) in accordance with user instructions.

In conjunction with touch screen 112, display system controller 156,contact module 130, graphics module 132, audio circuitry 110, speaker111, RF circuitry 108, and browser module 147, video and music playermodule 152 includes executable instructions that allow the user todownload and play back recorded music and other sound files stored inone or more file formats, such as MP3 or AAC files, and executableinstructions to display, present or otherwise play back videos (e.g., ontouch screen 112 or on an external, connected display via external port124). In some embodiments, device 100 optionally includes thefunctionality of an MP3 player, such as an iPod (trademark of AppleInc.).

In conjunction with touch screen 112, display controller 156, contactmodule 130, graphics module 132, and text input module 134, notes module153 includes executable instructions to create and manage notes, to dolists, and the like in accordance with user instructions.

In conjunction with RF circuitry 108, touch screen 112, display systemcontroller 156, contact module 130, graphics module 132, text inputmodule 134, GPS module 135, and browser module 147, map module 154 are,optionally, used to receive, display, modify, and store maps and dataassociated with maps (e.g., driving directions; data on stores and otherpoints of interest at or near a particular location; and otherlocation-based data) in accordance with user instructions.

In conjunction with touch screen 112, display system controller 156,contact module 130, graphics module 132, audio circuitry 110, speaker111, RF circuitry 108, text input module 134, e-mail client module 140,and browser module 147, online video module 155 includes instructionsthat allow the user to access, browse, receive (e.g., by streamingand/or download), play back (e.g., on the touch screen or on anexternal, connected display via external port 124), send an e-mail witha link to a particular online video, and otherwise manage online videosin one or more file formats, such as H.264. In some embodiments, instantmessaging module 141, rather than e-mail client module 140, is used tosend a link to a particular online video.

Each of the above identified modules and applications correspond to aset of executable instructions for performing one or more functionsdescribed above and the methods described in this application (e.g., thecomputer-implemented methods and other information processing methodsdescribed herein). These modules (e.g., sets of instructions) need notbe implemented as separate software programs, procedures or modules, andthus various subsets of these modules are, optionally, combined orotherwise re-arranged in various embodiments. In some embodiments,memory 102 optionally stores a subset of the modules and data structuresidentified above. Furthermore, memory 102 optionally stores additionalmodules and data structures not described above.

In some embodiments, device 100 is a device where operation of apredefined set of functions on the device is performed exclusivelythrough a touch screen and/or a touchpad (whether included in device 100or on a separate device, such as an input device). By using a touchscreen and/or a touchpad as the primary input control device foroperation of device 100, the number of physical input control devices(such as push buttons, dials, and the like) on device 100 is,optionally, reduced.

The predefined set of functions that are performed exclusively through atouch screen and/or a touchpad optionally include navigation betweenuser interfaces. In some embodiments, the touchpad, when touched by theuser, navigates device 100 to a main, home, or root menu from any userinterface that is displayed on device 100. In such embodiments, a “menubutton” is implemented using a touchpad. In some other embodiments, themenu button is a physical push button or other physical input controldevice instead of a touchpad.

FIG. 1B is a block diagram illustrating exemplary components for eventhandling in accordance with some embodiments. In some embodiments,memory 102 (in FIG. 1A) or 370 (FIG. 3 ) includes event sorter 170(e.g., in operating system 126) and a respective application 136-1(e.g., any of the aforementioned applications 137-151, 155, 380-390).

Event sorter 170 receives event information and determines theapplication 136-1 and application view 191 of application 136-1 to whichto deliver the event information. Event sorter 170 includes eventmonitor 171 and event dispatcher module 174. In some embodiments,application 136-1 includes application internal state 192, whichindicates the current application view(s) displayed on touch-sensitivedisplay 112 when the application is active or executing. In someembodiments, device/global internal state 157 is used by event sorter170 to determine which application(s) is (are) currently active, andapplication internal state 192 is used by event sorter 170 to determineapplication views 191 to which to deliver event information.

In some embodiments, application internal state 192 includes additionalinformation, such as one or more of: resume information to be used whenapplication 136-1 resumes execution, user interface state informationthat indicates information being displayed or that is ready for displayby application 136-1, a state queue for enabling the user to go back toa prior state or view of application 136-1, and a redo/undo queue ofprevious actions taken by the user.

Event monitor 171 receives event information from peripherals interface118. Event information includes information about a sub-event (e.g., auser touch on touch-sensitive display 112, as part of a multi-touchgesture). Peripherals interface 118 transmits information it receivesfrom I/O subsystem 106 or a sensor, such as proximity sensor 166,accelerometer(s) 168, and/or microphone 113 (through audio circuitry110). Information that peripherals interface 118 receives from I/Osubsystem 106 includes information from touch-sensitive display 112 or atouch-sensitive surface.

In some embodiments, event monitor 171 sends requests to the peripheralsinterface 118 at predetermined intervals. In response, peripheralsinterface 118 transmits event information. In other embodiments,peripheral interface 118 transmits event information only when there isa significant event (e.g., receiving an input above a predeterminednoise threshold and/or for more than a predetermined duration).

In some embodiments, event sorter 170 also includes a hit viewdetermination module 172 and/or an active event recognizer determinationmodule 173.

Hit view determination module 172 provides software procedures fordetermining where a sub-event has taken place within one or more views,when touch-sensitive display 112 displays more than one view. Views aremade up of controls and other elements that a user can see on thedisplay.

Another aspect of the user interface associated with an application is aset of views, sometimes herein called application views or userinterface windows, in which information is displayed and touch-basedgestures occur. The application views (of a respective application) inwhich a touch is detected optionally correspond to programmatic levelswithin a programmatic or view hierarchy of the application. For example,the lowest level view in which a touch is detected is, optionally,called the hit view, and the set of events that are recognized as properinputs are, optionally, determined based, at least in part, on the hitview of the initial touch that begins a touch-based gesture.

Hit view determination module 172 receives information related tosub-events of a touch-based gesture. When an application has multipleviews organized in a hierarchy, hit view determination module 172identifies a hit view as the lowest view in the hierarchy which shouldhandle the sub-event. In most circumstances, the hit view is the lowestlevel view in which an initiating sub-event occurs (e.g., the firstsub-event in the sequence of sub-events that form an event or potentialevent). Once the hit view is identified by the hit view determinationmodule, the hit view typically receives all sub-events related to thesame touch or input source for which it was identified as the hit view.

Active event recognizer determination module 173 determines which viewor views within a view hierarchy should receive a particular sequence ofsub-events. In some embodiments, active event recognizer determinationmodule 173 determines that only the hit view should receive a particularsequence of sub-events. In other embodiments, active event recognizerdetermination module 173 determines that all views that include thephysical location of a sub-event are actively involved views, andtherefore determines that all actively involved views should receive aparticular sequence of sub-events. In other embodiments, even if touchsub-events were entirely confined to the area associated with oneparticular view, views higher in the hierarchy would still remain asactively involved views.

Event dispatcher module 174 dispatches the event information to an eventrecognizer (e.g., event recognizer 180). In embodiments including activeevent recognizer determination module 173, event dispatcher module 174delivers the event information to an event recognizer determined byactive event recognizer determination module 173. In some embodiments,event dispatcher module 174 stores in an event queue the eventinformation, which is retrieved by a respective event receiver module182.

In some embodiments, operating system 126 includes event sorter 170.Alternatively, application 136-1 includes event sorter 170. In yet otherembodiments, event sorter 170 is a stand-alone module, or a part ofanother module stored in memory 102, such as contact/motion module 130.

In some embodiments, application 136-1 includes a plurality of eventhandlers 190 and one or more application views 191, each of whichincludes instructions for handling touch events that occur within arespective view of the application's user interface. Each applicationview 191 of the application 136-1 includes one or more event recognizers180. Typically, a respective application view 191 includes a pluralityof event recognizers 180. In other embodiments, one or more of eventrecognizers 180 are part of a separate module, such as a user interfacekit (not shown) or a higher level object from which application 136-1inherits methods and other properties. In some embodiments, a respectiveevent handler 190 includes one or more of: data updater 176, objectupdater 177, GUI updater 178, and/or event data 179 received from eventsorter 170. Event handler 190 optionally utilizes or calls data updater176, object updater 177 or GUI updater 178 to update the applicationinternal state 192. Alternatively, one or more of the application views191 includes one or more respective event handlers 190. Also, in someembodiments, one or more of data updater 176, object updater 177, andGUI updater 178 are included in a respective application view 191.

A respective event recognizer 180 receives event information (e.g.,event data 179) from event sorter 170, and identifies an event from theevent information. Event recognizer 180 includes event receiver 182 andevent comparator 184. In some embodiments, event recognizer 180 alsoincludes at least a subset of: metadata 183, and event deliveryinstructions 188 (which optionally include sub-event deliveryinstructions).

Event receiver 182 receives event information from event sorter 170. Theevent information includes information about a sub-event, for example, atouch or a touch movement. Depending on the sub-event, the eventinformation also includes additional information, such as location ofthe sub-event. When the sub-event concerns motion of a touch, the eventinformation optionally also includes speed and direction of thesub-event. In some embodiments, events include rotation of the devicefrom one orientation to another (e.g., from a portrait orientation to alandscape orientation, or vice versa), and the event informationincludes corresponding information about the current orientation (alsocalled device attitude) of the device.

Event comparator 184 compares the event information to predefined eventor sub-event definitions and, based on the comparison, determines anevent or sub-event, or determines or updates the state of an event orsub-event. In some embodiments, event comparator 184 includes eventdefinitions 186. Event definitions 186 contain definitions of events(e.g., predefined sequences of sub-events), for example, event 1(187-1), event 2 (187-2), and others. In some embodiments, sub-events inan event 187 include, for example, touch begin, touch end, touchmovement, touch cancellation, and multiple touching. In one example, thedefinition for event 1 (187-1) is a double tap on a displayed object.The double tap, for example, comprises a first touch (touch begin) onthe displayed object for a predetermined phase, a first lift-off (touchend) for a predetermined phase, a second touch (touch begin) on thedisplayed object for a predetermined phase, and a second lift-off (touchend) for a predetermined phase. In another example, the definition forevent 2 (187-2) is a dragging on a displayed object. The dragging, forexample, comprises a touch (or contact) on the displayed object for apredetermined phase, a movement of the touch across touch-sensitivedisplay 112, and lift-off of the touch (touch end). In some embodiments,the event also includes information for one or more associated eventhandlers 190.

In some embodiments, event definition 187 includes a definition of anevent for a respective user-interface object. In some embodiments, eventcomparator 184 performs a hit test to determine which user-interfaceobject is associated with a sub-event. For example, in an applicationview in which three user-interface objects are displayed ontouch-sensitive display 112, when a touch is detected on touch-sensitivedisplay 112, event comparator 184 performs a hit test to determine whichof the three user-interface objects is associated with the touch(sub-event). If each displayed object is associated with a respectiveevent handler 190, the event comparator uses the result of the hit testto determine which event handler 190 should be activated. For example,event comparator 184 selects an event handler associated with thesub-event and the object triggering the hit test.

In some embodiments, the definition for a respective event 187 alsoincludes delayed actions that delay delivery of the event informationuntil after it has been determined whether the sequence of sub-eventsdoes or does not correspond to the event recognizer's event type.

When a respective event recognizer 180 determines that the series ofsub-events do not match any of the events in event definitions 186, therespective event recognizer 180 enters an event impossible, eventfailed, or event ended state, after which it disregards subsequentsub-events of the touch-based gesture. In this situation, other eventrecognizers, if any, that remain active for the hit view continue totrack and process sub-events of an ongoing touch-based gesture.

In some embodiments, a respective event recognizer 180 includes metadata183 with configurable properties, flags, and/or lists that indicate howthe event delivery system should perform sub-event delivery to activelyinvolved event recognizers. In some embodiments, metadata 183 includesconfigurable properties, flags, and/or lists that indicate how eventrecognizers interact, or are enabled to interact, with one another. Insome embodiments, metadata 183 includes configurable properties, flags,and/or lists that indicate whether sub-events are delivered to varyinglevels in the view or programmatic hierarchy.

In some embodiments, a respective event recognizer 180 activates eventhandler 190 associated with an event when one or more particularsub-events of an event are recognized. In some embodiments, a respectiveevent recognizer 180 delivers event information associated with theevent to event handler 190. Activating an event handler 190 is distinctfrom sending (and deferred sending) sub-events to a respective hit view.In some embodiments, event recognizer 180 throws a flag associated withthe recognized event, and event handler 190 associated with the flagcatches the flag and performs a predefined process.

In some embodiments, event delivery instructions 188 include sub-eventdelivery instructions that deliver event information about a sub-eventwithout activating an event handler. Instead, the sub-event deliveryinstructions deliver event information to event handlers associated withthe series of sub-events or to actively involved views. Event handlersassociated with the series of sub-events or with actively involved viewsreceive the event information and perform a predetermined process.

In some embodiments, data updater 176 creates and updates data used inapplication 136-1. For example, data updater 176 updates the telephonenumber used in contacts module 137, or stores a video file used in videoplayer module 145. In some embodiments, object updater 177 creates andupdates objects used in application 136-1. For example, object updater176 creates a new user-interface object or updates the position of auser-interface object. GUI updater 178 updates the GUI. For example, GUIupdater 178 prepares display information and sends it to graphics module132 for display on a touch-sensitive display.

In some embodiments, event handler(s) 190 includes or has access to dataupdater 176, object updater 177, and GUI updater 178. In someembodiments, data updater 176, object updater 177, and GUI updater 178are included in a single module of a respective application 136-1 orapplication view 191. In other embodiments, they are included in two ormore software modules.

It shall be understood that the foregoing discussion regarding eventhandling of user touches on touch-sensitive displays and/or touchpadsalso applies to other forms of user inputs to operate multifunctiondevices 100 with input-devices, not all of which are initiated on touchscreens. For example, mouse movement and mouse button presses,optionally coordinated with single or multiple keyboard presses orholds; contact movements such as taps, drags, scrolls, etc., ontouch-pads; pen stylus inputs; movement of the device; oralinstructions; detected eye movements; biometric inputs; and/or anycombination thereof are optionally utilized as inputs corresponding tosub-events which define an event to be recognized.

FIG. 2 illustrates a portable or non-portable multifunction device 100having a touch screen 112 in accordance with some embodiments. As statedabove, multifunction device 100 is described as having the variousillustrated structures (such as touch screen 112, speaker 111,accelerometer 168, microphone 113, etc.); however, it is understood thatthese structures optionally reside on separate devices. For example,display-related structures (e.g., display, speaker, etc.) and/orfunctions optionally reside on a separate display device, input-relatedstructures (e.g., touch-sensitive surface, microphone, accelerometer,etc.) and/or functions optionally reside on a separate input device, andremaining structures and/or functions optionally reside on multifunctiondevice 100.

The touch screen 112 optionally displays one or more graphics withinuser interface (UI) 200. In this embodiment, as well as others describedbelow, a user is enabled to select one or more of the graphics by makinga gesture on the graphics, for example, with one or more fingers 202(not drawn to scale in the figure) or one or more styluses 203 (notdrawn to scale in the figure). In some embodiments, selection of one ormore graphics occurs when the user breaks contact with the one or moregraphics. In some embodiments, the gesture optionally includes one ormore taps, one or more swipes (from left to right, right to left, upwardand/or downward) and/or a rolling of a finger (from right to left, leftto right, upward and/or downward) that has made contact with device 100.In some implementations or circumstances, inadvertent contact with agraphic does not select the graphic. For example, a swipe gesture thatsweeps over an application icon optionally does not select thecorresponding application when the gesture corresponding to selection isa tap.

Device 100 optionally also includes one or more physical buttons, suchas “home” or menu button 204. As described previously, menu button 204is, optionally, used to navigate to any application 136 in a set ofapplications that are, optionally executed on device 100. Alternatively,in some embodiments, the menu button is implemented as a soft key in aGUI displayed on touch screen 112.

In one embodiment, device 100 includes touch screen 112, menu button204, push button 206 for powering the device on/off and locking thedevice, volume adjustment button(s) 208, Subscriber Identity Module(SIM) card slot 210, head set jack 212, and docking/charging externalport 124. Push button 206 is, optionally, used to turn the power on/offon the device by depressing the button and holding the button in thedepressed state for a predefined time interval; to lock the device bydepressing the button and releasing the button before the predefinedtime interval has elapsed; and/or to unlock the device or initiate anunlock process. In an alternative embodiment, device 100 also acceptsverbal input for activation or deactivation of some functions throughmicrophone 113. Device 100 also, optionally, includes one or morecontact intensity sensors 165 for detecting intensity of contacts ontouch screen 112 and/or one or more tactile output generators 167 forgenerating tactile outputs for a user of device 100.

FIG. 3 is a block diagram of an exemplary multifunction device with adisplay and a touch-sensitive surface in accordance with someembodiments. Device 300 need not include the display and thetouch-sensitive surface, as described above, but rather, in someembodiments, optionally communicates with the display and thetouch-sensitive surface on other devices. Additionally, device 300 neednot be portable. In some embodiments, device 300 is a laptop computer, adesktop computer, a tablet computer, a multimedia player device (such asa television or a set-top box), a navigation device, an educationaldevice (such as a child's learning toy), a gaming system, or a controldevice (e.g., a home or industrial controller). Device 300 typicallyincludes one or more processing units (CPU's) 310, one or more networkor other communications interfaces 360, memory 370, and one or morecommunication buses 320 for interconnecting these components.Communication buses 320 optionally include circuitry (sometimes called achipset) that interconnects and controls communications between systemcomponents. Device 300 includes input/output (I/O) interface 330comprising display 340, which is typically a touch screen display. I/Ointerface 330 also optionally includes a keyboard and/or mouse (or otherpointing device) 350 and touchpad 355, tactile output generator 357 forgenerating tactile outputs on device 300 (e.g., similar to tactileoutput generator(s) 167 described above with reference to FIG. 1A),sensors 359 (e.g., optical, acceleration, proximity, touch-sensitive,and/or contact intensity sensors similar to contact intensity sensor(s)165 described above with reference to FIG. 1A). Memory 370 includeshigh-speed random access memory, such as DRAM, SRAM, DDR RAM or otherrandom access solid state memory devices; and optionally includesnon-volatile memory, such as one or more magnetic disk storage devices,optical disk storage devices, flash memory devices, or othernon-volatile solid state storage devices. Memory 370 optionally includesone or more storage devices remotely located from CPU(s) 310. In someembodiments, memory 370 stores programs, modules, and data structuresanalogous to the programs, modules, and data structures stored in memory102 of portable or non-portable multifunction device 100 (FIG. 1A), or asubset thereof. Furthermore, memory 370 optionally stores additionalprograms, modules, and data structures not present in memory 102 ofportable or non-portable multifunction device 100. For example, memory370 of device 300 optionally stores drawing module 380, presentationmodule 382, word processing module 384, website creation module 386,disk authoring module 388, and/or spreadsheet module 390, while memory102 of portable or non-portable multifunction device 100 (FIG. 1A)optionally does not store these modules.

Each of the above identified elements in FIG. 3 are, optionally, storedin one or more of the previously mentioned memory devices. Each of theabove identified modules corresponds to a set of instructions forperforming a function described above. The above identified modules orprograms (e.g., sets of instructions) need not be implemented asseparate software programs, procedures or modules, and thus varioussubsets of these modules are, optionally, combined or otherwisere-arranged in various embodiments. In some embodiments, memory 370optionally stores a subset of the modules and data structures identifiedabove. Furthermore, memory 370 optionally stores additional modules anddata structures not described above.

FIG. 4 illustrates an exemplary user interface on a device (e.g., device300, FIG. 3 ) with a touch-sensitive surface 451 (e.g., a tablet ortouchpad 355, FIG. 3 ) that is separate from the display 450 (e.g.,touch screen display 112). Device 300 also, optionally, includes one ormore contact intensity sensors (e.g., one or more of sensors 357) fordetecting intensity of contacts on touch-sensitive surface 451 and/orone or more tactile output generators 359 for generating tactile outputsfor a user of device 300.

Although some of the examples which follow will be given with referenceto inputs on touch screen display 112 (where the touch sensitive surfaceand the display are combined), in some embodiments, the device detectsinputs on a touch-sensitive surface that is separate from the display,as shown in FIG. 4 . In some embodiments the touch sensitive surface(e.g., 451 in FIG. 4 ) has a primary axis (e.g., 452 in FIG. 4 ) thatcorresponds to a primary axis (e.g., 453 in FIG. 4 ) on the display(e.g., 450). In accordance with these embodiments, the device detectscontacts (e.g., 460 and 462 in FIG. 4 ) with the touch-sensitive surface451 at locations that correspond to respective locations on the display(e.g., in FIG. 4, 460 corresponds to 468 and 462 corresponds to 470). Inthis way, user inputs (e.g., contacts 460 and 462, and movementsthereof) detected by the device on the touch-sensitive surface (e.g.,451 in FIG. 4 ) are used by the device to manipulate the user interfaceon the display (e.g., 450 in FIG. 4 ) of the multifunction device whenthe touch-sensitive surface is separate from the display. It should beunderstood that similar methods are, optionally, used for other userinterfaces described herein.

Additionally, while the following examples are given primarily withreference to finger inputs (e.g., finger contacts, finger tap gestures,finger swipe gestures), it should be understood that, in someembodiments, one or more of the finger inputs are replaced with inputfrom another input device (e.g., a mouse based input or stylus input).For example, a swipe gesture is, optionally, replaced with a mouse click(e.g., instead of a contact) followed by movement of the cursor alongthe path of the swipe (e.g., instead of movement of the contact). Asanother example, a tap gesture is, optionally, replaced with a mouseclick while the cursor is located over the location of the tap gesture(e.g., instead of detection of the contact followed by ceasing to detectthe contact). Similarly, when multiple user inputs are simultaneouslydetected, it should be understood that multiple computer mice are,optionally, used simultaneously, or a mouse and finger contacts are,optionally, used simultaneously.

As used herein, the term “focus selector” refers to an input elementthat indicates a current part of a user interface with which a user isinteracting. In some implementations that include a cursor or otherlocation marker, the cursor acts as a “focus selector,” so that when aninput (e.g., a press input) is detected on a touch-sensitive surface(e.g., touchpad 355 in FIG. 3 or touch-sensitive surface 451 in FIG. 4 )while the cursor is over a particular user interface element (e.g., abutton, window, slider or other user interface element), the particularuser interface element is adjusted in accordance with the detectedinput. In some implementations that include a touch-screen display(e.g., touch-sensitive display system 112 in FIG. 1A or touch screen 112in FIG. 4A) that enables direct interaction with user interface elementson the touch-screen display, a detected contact on the touch-screen actsas a “focus selector,” so that when an input (e.g., a press input by thecontact) is detected on the touch-screen display at a location of aparticular user interface element (e.g., a button, window, slider orother user interface element), the particular user interface element isadjusted in accordance with the detected input. In some implementationsfocus is moved from one region of a user interface to another region ofthe user interface without corresponding movement of a cursor ormovement of a contact on a touch-screen display (e.g., by using a tabkey or arrow keys to move focus from one button to another button); inthese implementations, the focus selector moves in accordance withmovement of focus between different regions of the user interface.Without regard to the specific form taken by the focus selector, thefocus selector is generally the user interface element (or contact on atouch-screen display) that is controlled by the user so as tocommunicate the user's intended interaction with the user interface(e.g., by indicating, to the device, the element of the user interfacewith which the user is intending to interact). For example, the locationof a focus selector (e.g., a cursor, a contact or a selection box) overa respective button while a press input is detected on thetouch-sensitive surface (e.g., a touchpad or touch screen) will indicatethat the user is intending to activate the respective button (as opposedto other user interface elements shown on a display of the device).

In some embodiments described herein, one or more operations areperformed in response to detecting a gesture that includes a respectivepress input or in response to detecting the respective press inputperformed with a respective contact (or a plurality of contacts), wherethe respective press input is detected based at least in part ondetecting an increase in intensity of the contact (or plurality ofcontacts) above a press-input intensity threshold. In some embodiments,the respective operation is performed in response to detecting theincrease in intensity of the respective contact above the press-inputintensity threshold (e.g., a “down stroke” of the respective pressinput). In some embodiments, the press input includes an increase inintensity of the respective contact above the press-input intensitythreshold and a subsequent decrease in intensity of the contact belowthe press-input intensity threshold, and the respective operation isperformed in response to detecting the subsequent decrease in intensityof the respective contact below the press-input threshold (e.g., an “upstroke” of the respective press input).

In some embodiments, the device employs intensity hysteresis to avoidaccidental inputs sometimes termed “jitter,” where the device defines orselects a hysteresis intensity threshold with a predefined relationshipto the press-input intensity threshold (e.g., the hysteresis intensitythreshold is X intensity units lower than the press-input intensitythreshold or the hysteresis intensity threshold is 75%, 90% or somereasonable proportion of the press-input intensity threshold). Thus, insome embodiments, the press input includes an increase in intensity ofthe respective contact above the press-input intensity threshold and asubsequent decrease in intensity of the contact below the hysteresisintensity threshold that corresponds to the press-input intensitythreshold, and the respective operation is performed in response todetecting the subsequent decrease in intensity of the respective contactbelow the hysteresis intensity threshold (e.g., an “up stroke” of therespective press input). Similarly, in some embodiments, the press inputis detected only when the device detects an increase in intensity of thecontact from an intensity at or below the hysteresis intensity thresholdto an intensity at or above the press-input intensity threshold and,optionally, a subsequent decrease in intensity of the contact to anintensity at or below the hysteresis intensity, and the respectiveoperation is performed in response to detecting the press input (e.g.,the increase in intensity of the contact or the decrease in intensity ofthe contact, depending on the circumstances).

For ease of explanation, the description of operations performed inresponse to a press input associated with a press-input intensitythreshold or in response to a gesture including the press input are,optionally, triggered in response to detecting either: an increase inintensity of a contact above the press-input intensity threshold, anincrease in intensity of a contact from an intensity below thehysteresis intensity threshold to an intensity above the press-inputintensity threshold, a decrease in intensity of the contact below thepress-input intensity threshold, and/or a decrease in intensity of thecontact below the hysteresis intensity threshold corresponding to thepress-input intensity threshold. Additionally, in examples where anoperation is described as being performed in response to detecting adecrease in intensity of a contact below the press-input intensitythreshold, the operation is, optionally, performed in response todetecting a decrease in intensity of the contact below a hysteresisintensity threshold corresponding to, and lower than, the press-inputintensity threshold.

FIG. 5 illustrates a block diagram of an exemplary architecture for thedevice 500 according to some embodiments of the disclosure. In theembodiment of FIG. 5 , media content is optionally received by device500 via network interface 502, which is optionally a wireless or wiredconnection. The one or more processors 504 optionally execute any numberof programs stored in memory 506 or storage, which optionally includesinstructions to perform one or more of the methods and/or processesdescribed herein (e.g., methods 700, 900, 1100, 1300, and/or 1500).

In some embodiments, display controller 508 causes the various userinterfaces of the disclosure to be displayed on display 500. Further,input to device 500 is optionally provided by remote 510 via remoteinterface 512, which is optionally a wireless or a wired connection. Itis understood that the embodiment of FIG. 5 is not meant to limit thefeatures of the device of the disclosure, and that other components tofacilitate other features described in the disclosure are optionallyincluded in the architecture of FIG. 5 as well. In some embodiments,device 500 optionally corresponds to one or more of multifunction device100 in FIGS. 1A and 2 and device 300 in FIG. 3 ; network interface 502optionally corresponds to one or more of RF circuitry 108, external port124, and peripherals interface 118 in FIGS. 1A and 2 , and networkcommunications interface 360 in FIG. 3 ; processor 504 optionallycorresponds to one or more of processor(s) 120 in FIG. 1A and CPU(s) 310in FIG. 3 ; display controller 508 optionally corresponds to one or moreof display controller 156 in FIG. 1A and I/O interface 330 in FIG. 3 ;memory 506 optionally corresponds to one or more of memory 102 in FIG.1A and memory 370 in FIG. 3 ; remote interface 512 optionallycorresponds to one or more of peripherals interface 118, and I/Osubsystem 106 (and/or its components) in FIG. 1A, and I/O interface 330in FIG. 3 ; remote 512 optionally corresponds to and or includes one ormore of speaker 111, touch-sensitive display system 112, microphone 113,optical sensor(s) 164, contact intensity sensor(s) 165, tactile outputgenerator(s) 167, other input control devices 116, accelerometer(s) 168,proximity sensor 166, and I/O subsystem 106 in FIG. 1A, andkeyboard/mouse 350, touchpad 355, tactile output generator(s) 357, andcontact intensity sensor(s) 359 in FIG. 3 , and touch-sensitive surface451 in FIG. 4 ; and, display 514 optionally corresponds to one or moreof touch-sensitive display system 112 in FIGS. 1A and 2 , and display340 in FIG. 3 .

User Interfaces and Associated Processes Facial Identification

People interact with electronic devices in many ways, such as runningapplications and performing various tasks. These interactions can beenhanced by identifying the faces of the people interacting with thedevices and performing various actions in accordance with theidentifications—in some cases, without requiring additional input fromthe people. The embodiments described below provide ways to enhance theabove interactions. Enhancing interactions with a device reduces theamount of time needed by a person to perform operations, and thusreduces the power usage of the device and increases battery life forbattery-powered devices.

FIGS. 6A-6D illustrate exemplary ways in which interactions withelectronic devices are enhanced based on facial identification inaccordance with some embodiments of the disclosure. The embodiments inthese figures are used to illustrate the processes described below,including the processes described with reference to FIGS. 7A-7E.

FIG. 6A illustrates an exemplary use configuration for electronic device600 according to embodiments of the disclosure. In some embodiments,electronic device 600 is optionally device 500 as described above withreference to FIG. 5 , including a set top box or other user interfacegenerating device that is in communication with a remote control and adisplay device. Optionally, electronic device 600 is coupled to a camera602. Camera 602 is optionally included in electronic device 600; in someembodiments, the camera is external to the electronic device, forexample, in a display device to which the electronic device is coupled.In some embodiments, camera 602 optionally corresponds to any one ofoptical sensor(s) 164 in FIG. 1A and sensors (359) in FIG. 3 . Camera602 optionally includes any type of image sensor, such as ared-green-blue (RGB) image sensor, a depth image sensor, and an infrared(IR) image sensor. Electronic device 600 optionally receives sensedimage data from camera 602. In the embodiment illustrated, a firstperson 604 is located in front of camera 602 such that the camera isable to capture an image of the first person. It is understood thatpeople use devices. When a person uses a device (e.g., device 600), thatperson is optionally referred to as a user of the device. Thus, in thisdisclosure, references to a “person” or “people” are optionallyinterchangeable with references to a “user” or “users”, and referencesto a “user” or “users” are optionally interchangeable with references toa “person” or “persons”.

FIG. 6B illustrates an exemplary first image 606 provided by camera 602to electronic device 600. In some embodiments, first image 606 includesfirst person 604 and other people or things 610. In some embodiments,electronic device 600 identifies the face 608 of first person 604 fromfirst image 606. Identifying the face 608 of first person 604 optionallyincludes determining the shape and/or location of the face in the image,and not necessarily the identity of the person to whom the face belongs,though both are within the scope of the disclosure. Electronic device600 optionally performs an action in accordance with the identificationof the face 608 of first person 604, as will be described in more detailbelow.

FIG. 6C illustrates an exemplary action performed by electronic device600 in accordance with the identification of the face 608 of firstperson 604. In some embodiments, electronic device 600 generates a userinterface for display on a display device, such as a television. In someembodiments, the user interface includes second image 612 that includesthe face 614 of a second person 616, where the face 614 of the secondperson 616 is different from the face 608 of first person 604 (e.g.,second image 612 is an image of a person different than first person604). Electronic device 600 optionally identifies the face 614 of thesecond person 616 from second image 612. Identifying the face 614 of thesecond person 616 optionally includes determining the shape and/orlocation of the face in the image, and not necessarily the identity ofthe person to whom the face belongs, though both are within the scope ofthe disclosure. In the illustrated embodiment, electronic device 600replaces the face 614 of the second person 616 in second image 612 withthe face 608 of first person 604 from first image 606, thus providing asimple and automatic way of replacing one person's face with anotherperson's face in an image.

FIG. 6D illustrates another exemplary action performed by electronicdevice 600 in accordance with the identification of the face 608 offirst person 604. As described above, in some embodiments, electronicdevice 600 generates a user interface 618 for display on a displaydevice, such as a television. User interface 618 optionally includes oneor more user interface elements 620, 622 and 624, which optionallyinclude things such as images, buttons, toolbars, media content, icons,application windows, etc. Electronic device 600 optionally determines anorientation of face 608 of first person 604 in first image 606 withrespect to the display device. Determining an orientation of face 608 offirst person 604 with respect to the display device optionally includesdetermining whether first person's 604 face 608 aligns with the displaydevice (e.g., the top of the person's face aligns with the top of thedisplay device, the bottom of the person's face aligns with the bottomof the display device, etc.). In some embodiments, electronic device 600aligns user interface 618 for display on the display device with theorientation of face 608 of first person 604 in first image 606. Aligninguser interface 618 optionally includes lining up the user interface withthe first person's 604 face 608 by, for example, rotating a userinterface element (e.g., image, toolbar, media content, icon,application window, etc.) such that one or more of thetop/bottom/left/right of the user interface element align with one ormore of the top/bottom/left/right of the face 608 of the first person604 in the first image 606, as illustrated.

In some embodiments, aligning user interface 618 with the first person's604 face 608 includes forgoing aligning user interface 618 based on anorientation of the electronic device 600 detected by one or more sensorsof the electronic device. For example, even if a mobile phone is beingheld sideways with respect to gravity as detected by one or moreaccelerometers or an inertial measurement unit, the mobile phoneoptionally does not rotate the user interface on the mobile phone if theperson in front of the mobile phone is also sideways, e.g., looking atthe phone while laying down on his/her side. For example, in somecircumstances, the default behavior of the device optionally is to alignthe user interface so that a bottom edge of the user interface isaligned with the edge of the device that is oriented down (e.g., closestto the floor), but if a face is detected, this default behavior isoptionally overridden so that even if a first edge of the device isclosest to the floor, the user interface will optionally be aligned witha second edge that causes the user interface to be oriented so that itappears right-side-up for the user.

In some embodiments, the user interface includes a content item beingplayed (e.g., a movie, a television show, a game, a song, a radio show),and performing the action on the electronic device in accordance withthe identification of the face of the first person optionally includesassociating the playing of the content item with the first person (e.g.,tracking the person's viewing or other activity history withoutrequiring the person to manually login to the electronic device, ormarking notifications as read by the user only if the user is in frontof the display when the notifications are displayed on the display, evenif the user is logged in and the device is not asleep). In someembodiments, the electronic device identifies, based on the face of thefirst person in the first image, a first profile on the electronicdevice that is associated with the face of the first person (e.g.,actually identifying the person). In such embodiments, associating theplaying of the content item with the first person optionally includesassociating the playing of the content item with the first profile. Insome embodiments, the electronic device updates a play count of thecontent item in the first profile.

In some embodiments, two or more people might be captured in an imagetaken by the camera. In such instances, the first image optionallyfurther includes a face of a second person. In addition to identifyingthe face of the first person in the first image, the electronic deviceoptionally identifies the face of the second person from the firstimage, identifies, based on the face of the second person in the firstimage, a second profile on the electronic device that is associated withthe face of the second person (e.g., actually identifying the secondperson), and associates the playing of the content item with the secondprofile. In other words, the electronic device optionally associates theplaying of the content item with both the first person and the secondperson.

In some embodiments, the second person might join the first person afterthe first person has started playing the content item. In suchinstances, the electronic device optionally receives a second imageincluding the face of the first person (e.g., a second image includingjust one person), identifies the face of the first person from thesecond image, and in response to identifying the face of the firstperson from the second image, continues to associate the playing of thecontent item with the first person (e.g., while the face of the firstperson continues to be detected, continuing to associate the playing ofthe content item with the first person). The electronic deviceoptionally receives a third image including the face of the first personand a face of a second person (e.g., a third image including both thefirst person and the second person), where the third image was capturedafter the second image was captured (e.g., the second person arrives atthe display device while the first person is watching the content item).In some embodiments, the electronic device identifies the face of thefirst person and the face of the second person from the third image(e.g., identifying both people at the display device), and in responseto identifying the face of the first person and the face of the secondperson from the third image, continues to associate the playing of thecontent item with the first person (e.g., while the face of the firstperson continues to be detected, continuing to associate the playing ofthe content item with the first person), and associates the playing ofthe content item with the second person (e.g., starts associating theplaying of the content item with the second person when the face of thesecond person is identified).

In some embodiments, the electronic device verifies that the firstperson is identified for longer than a certain time period beforefinally associating the playing of the content item with the firstperson. In such instances, as described above, the user interfacegenerated by the electronic device optionally includes a content itembeing played (e.g., a movie, a television show, a game, a song, a radioshow). Performing the action on the electronic device in accordance withthe identification of the face of the first person optionally includesmaking a preliminarily association of the playing of the content itemwith the first person (e.g., tracking the person's viewing or otheractivity history without requiring the person to manually login to theelectronic device, or marking notifications as read by the user only ifthe user is in front of the display when the notifications are displayedon the display, even if the user is logged in and the device is notasleep). In some embodiments, the electronic device receives a secondimage at least a first time threshold (e.g., 1, 15, or 30 seconds) afterreceiving the first image. After receiving the second image, theelectronic device optionally determines whether the second imageincludes the face of the first person. In accordance with adetermination that the second image does not include the face of thefirst person, the electronic device optionally cancels the preliminaryassociation of the playing of the content item with the first person. Inaccordance with a determination that the second image includes the faceof the first person, the electronic device optionally confirms thepreliminary association of the playing of the content item with thefirst person (e.g., to ensure that first person is “present” for longenough to associate the content item with that person).

FIGS. 7A-7E are flow diagrams illustrating a method 700 of performing anaction based in part on identification of a face in an image inaccordance with some embodiments (e.g., as described above withreference to FIGS. 6A-6D). The method 700 is optionally performed at anelectronic device such as device 500 as described above with referenceto FIG. 5 , including a set top box or other user interface generatingdevice that is in communication with a remote control and a displaydevice. Some operations in method 700 are, optionally, combined and/orthe order of some operations is, optionally, changed.

As described below, the method 700 provides ways in which a device canreact to identifying the face of a user. The method reduces thecognitive burden on a user when interacting with a user interface on thedevice, and reduces the need for the user to physically interact withthe device, thereby creating a more efficient human-machine interface.For battery-operated electronic devices, increasing the efficiency ofthe user's interaction with the user interface conserves power andincreases the time between battery charges.

In some embodiments, an electronic device (e.g., a set top box or otheruser interface generating device that is in communication with a remotecontrol and a display device, such as device 600 in FIG. 6A) with one ormore processors and memory generates (702) a user interface for displayon a display device (e.g., a television or other display device). Insome embodiments, the user interface includes a content item beingplayed (704) (e.g., a movie, a television show, a game, a song, a radioshow).

In some embodiments, the device receives (706) a first image including aface of a first person (e.g., an image of the person from an RGB camera,such as image 606 including face 608 in FIG. 6B). In some embodiments,the user interface includes a second image that includes a face of asecond person, the face of the second person being different from theface of the first person (708) (e.g., an image of someone other than thefirst person, such as image 612 including face 614 in FIG. 6C). In someembodiments, the first image further includes a face of a second person(710).

In some embodiments, the device identifies (712) the face of the firstperson from the first image (e.g., determining the shape/location of theface in the image, not necessarily the identity of the person to whomthe face belongs).

In some embodiments, the device performs (714) an action on theelectronic device in accordance with the identification of the face ofthe first person (e.g., displaying the image on the display device witheverything but the person's face having some visual effect, logging theperson into the electronic device, etc.). In some embodiments,performing the action on the electronic device in accordance with theidentification of the face of the first person comprises associating theplaying of the content item with the first person (716) (e.g., trackingthe person's viewing or other activity history without requiring theperson to manually login to the electronic device or markingnotifications as read by the user only if the user is in front of thedisplay when the notifications are displayed on the display, even if theuser is logged in and the device is not asleep). In some embodiments,performing the action on the electronic device in accordance with theidentification of the face of the first person comprises making apreliminarily association of the playing of the content item with thefirst person (718) (e.g., preliminarily tracking the person's viewing orother activity history without requiring the person to manually login tothe electronic device or marking notifications as read by the user onlyif the user is in front of the display when the notifications aredisplayed on the display, even if the user is logged in and the deviceis not asleep).

In some embodiments, the device identifies (720) the face of the secondperson from the second image (e.g., determining the shape/location ofthe face in the image, not necessarily the identity of the person towhom the face belongs). In some embodiments, performing the action onthe electronic device in accordance with the identification of the faceof the first person comprises replacing the face of the second person inthe second image with the face of the first person in the first image(722) (e.g., placing the first person's face into an image that hasanother face, such as replacing face 614 in image 612 with face 608 inFIG. 6C).

In some embodiments, the device determines (724) an orientation of theface of the first person in the first image with respect to the displaydevice (e.g., determining whether the first person's face aligns withthe display device). In some embodiments, performing the action on theelectronic device in accordance with the identification of the face ofthe first person comprises aligning the user interface for display onthe display device with the orientation of the face of the first personin the first image (726) (e.g., lining up the user interface with theperson's face by, for example, rotating a user interface element (e.g.,image, toolbar, icon, application window, etc.) such that one or more ofthe top/bottom/left/right of the user interface element align with oneor more of the top/bottom/left/right of the face of the first person inthe first image). FIG. 6D and its associated discussion provide anexemplary embodiment in which such aligning is performed. In someembodiments, aligning the user interface with the orientation of theface of the first person comprises forgoing aligning the user interfacebased on an orientation of the device detected by one or more sensors ofthe device (728) (e.g., even if a mobile phone is being held sidewayswith respect to gravity as detected by one or more accelerometers or aninertial measurement unit, the mobile phone not rotate the userinterface on the mobile phone if the person in front of the mobile phoneis also sideways, e.g., looking at the phone while laying down onhis/her side). For example, the default behavior of the device isoptionally to align the user interface so that a bottom edge of the userinterface is aligned with the edge of the device that is oriented down(e.g., closest to the floor), but if a face is detected, this defaultbehavior is overridden so that even if a first edge of the device isclosest to the floor, the user interface will be aligned with a secondedge that causes the user interface to be oriented so that it appearsright-side-up for a user.

In some embodiments, the device identifies (730), based on the face ofthe first person in the first image, a first profile on the electronicdevice that is associated with the face of the first person (e.g.,actually identifying the person). In some embodiments, associating theplaying of the content item with the first person comprises associatingthe playing of the content item with the first profile (732). In someembodiments, associating the playing of the content item with the firstprofile comprises updating a play count of the content item in the firstprofile (734).

In some embodiments, the device identifies (738) the face of the secondperson from the first image. In some embodiments, the device identifies(740), based on the face of the second person in the first image, asecond profile on the electronic device that is associated with the faceof the second person (e.g., identify the second user). In someembodiments, the device associates (742) the playing of the content itemwith the second profile.

In some embodiments, the device receives (746) a second image includingthe face of the first person (e.g., a second image including just oneperson). In some embodiments, the device identifies (748) the face ofthe first person from the second image. In some embodiments, in responseto identifying the face of the first person from the second image, thedevice continues to associate (750) the playing of the content item withthe first person (e.g., while the face of the first person continues tobe detected, continue to associate the playing of the content item withthe first person). In some embodiments, the device receives (752) athird image including the face of the first person and a face of asecond person (e.g., a third image including two people). In someembodiments, the third image was captured after the second image wascaptured (754) (e.g., the second person arrives at the display devicewhile the first person is watching the content item). In someembodiments, the device identifies (756) the face of the first personand the face of the second person from the third image (e.g.,identifying both people at the display device). In some embodiments, inresponse to identifying the face of the first person and the face of thesecond person from the third image (758) the device continues toassociate (760) the playing of the content item with the first person(e.g., while the face of the first person continues to be detected,continue to associate the playing of the content item with the firstperson), and the device associates (762) the playing of the content itemwith the second person (e.g., start associating the playing of thecontent item with the second person when the face of the second personis identified).

In some embodiments, the device receives (766) a second image. In someembodiments, the second image is received at least a first timethreshold after receiving the first image (768) (e.g., 1, 15, or 30seconds). In some embodiments, after receiving the second image, thedevice determines (770) whether the second image includes the face ofthe first person. In some embodiments, in accordance with adetermination that the second image does not include the face of thefirst person, the device cancels (772) the preliminary association ofthe playing of the content item with the first person. In someembodiments, in accordance with a determination that the second imageincludes the face of the first person, the device confirms (774) thepreliminary association of the playing of the content item with thefirst person (e.g., ensure that first person is “present” for longenough to associate the content item with them).

It should be understood that the particular order in which theoperations in FIGS. 7A-7E have been described is merely exemplary and isnot intended to indicate that the described order is the only order inwhich the operations could be performed. One of ordinary skill in theart would recognize various ways to reorder the operations describedherein. Additionally, it should be noted that details of other processesdescribed herein with respect to other methods described herein (e.g.,methods 900, 1100, 1300 and 1500) are also applicable in an analogousmanner to method 700 described above with respect to FIGS. 7A-7E. Forexample, the user interfaces, display device and content item describedabove with reference to method 700 optionally have one or more of thecharacteristics of the user interfaces, display devices and contentitems (or media content) described herein with reference to othermethods described herein (e.g., methods 900, 1100, 1300 and 1500). Forbrevity, these details are not repeated here.

The operations in the information processing methods described aboveare, optionally implemented by running one or more functional modules inan information processing apparatus such as general purpose processors(e.g., as described above with respect to FIGS. 1A, 3 and 5 ) orapplication specific chips.

The operations described above with reference to FIGS. 7A-7E are,optionally, implemented by components depicted in FIGS. 1A-1B. Forexample, generation operation 702, receiving operation 706,identification operation 712 and performing operation 714 are,optionally implemented by event sorter 170, event recognizer 180, andevent handler 190. Event monitor 171 in event sorter 170 detects acontact on touch-sensitive display 112, and event dispatcher module 174delivers the event information to application 136-1. A respective eventrecognizer 180 of application 136-1 compares the event information torespective event definitions 186, and determines whether a first contactat a first location on the touch-sensitive surface corresponds to apredefined event or sub-event, such as selection of an object on a userinterface. When a respective predefined event or sub-event is detected,event recognizer 180 activates an event handler 190 associated with thedetection of the event or sub-event. Event handler 190 optionallyutilizes or calls data updater 176 or object updater 177 to update theapplication internal state 192. In some embodiments, event handler 190accesses a respective GUI updater 178 to update what is displayed by theapplication. Similarly, it would be clear to a person having ordinaryskill in the art how other processes can be implemented based on thecomponents depicted in FIGS. 1A-1B.

Gaze Determination

Determining where in a user interface a person is looking (e.g., gazelocation), and performing one or more actions based on that gazelocation, can enhance the person's experience when interacting with theuser interface. Such a user interface, generated by a device, enablesthe person to perform certain actions associated with the device withoutthe need to provide physical input, such as through a touch-sensitivesurface or a mouse. The embodiments described below describe ways inwhich determining a person's gaze location in a user interface canenhance that person's experience when interacting with the userinterface. Enhancing the usability of a device reduces the amount oftime needed by the person to perform operations, and thus reduces thepower usage of the device and increases battery life for battery-powereddevices.

FIGS. 8A-8E illustrate exemplary ways in which interactions with userinterfaces are enhanced based on gaze determination in accordance withsome embodiments of the disclosure. The embodiments in these figures areused to illustrate the processes described below, including theprocesses described with reference to FIGS. 9A-9C.

FIG. 8A illustrates an exemplary user interface 800 according toembodiments of the disclosure. User interface 800 is optionallygenerated by an electronic device such as device 500 as described abovewith reference to FIG. 5 , including a set top box or other userinterface generating device that is in communication with a remotecontrol and a display device. User interface 800 is optionally fordisplay on a display device, such as a television or other displaydevice. In the illustrated embodiment, user interface 800 includes aplurality of user interface elements 802, 804, 806, 808 and 810. Theuser interface elements are optionally any combination of any userinterface elements that can appear in a user interface, such as icons,application windows, buttons, toolbars, media content, docks, etc.Though user interface 800 includes a plurality of user interfaceelements, it is understood that the processes described here areoptionally performed in a user interface including only a single userinterface element.

The electronic device described above optionally determines a gazelocation 812 in user interface 800. The gaze location 812 is a locationin user interface 800 to which a person's gaze is directed. In otherwords, the electronic device optionally determines where in userinterface 800 the person is looking. In making this determination, theelectronic device uses any suitable method for determining the gazelocation 812, such as identifying the person's eyes and their direction,and the person's distance from the display device, and calculating thegaze location based on those quantities. Other methods of determiningthe gaze location 812 are similarly within the scope of the disclosure.

Once the electronic device determines the gaze location 812, theelectronic device optionally determines whether the gaze location 812coincides with a user interface element in user interface 800. In otherwords, the electronic device optionally determines whether the person islooking at a user interface element in user interface 800. In theillustrated embodiment, if the electronic device were determiningwhether the gaze location 812 coincides with user interface element 806,for example, the electronic device would determine that it does not. If,however, the electronic devise were determining whether the gazelocation 812 coincides with user interface element 810, the electronicdevice would determine that is does.

In accordance with a determination that the gaze location 812 coincideswith user interface element 810 (e.g., a determination that the personis looking at user interface element 810), the electronic deviceoptionally performs an action with respect to user interface element810. Possible actions that the electronic device optionally performs aredescribed in more detail below.

FIGS. 8B-8C illustrate an exemplary action that the electronic deviceoptionally performs based on the gaze location 812 in user interface800. FIG. 8B illustrates user interface 800, mostly as described above.However, in the embodiment of FIG. 8B, user interface element 806 isrequesting the person's attention by, for example, flashing, bouncing,or any other mechanism for requesting the person's attention. Userinterface element 806 might request the person's attention for manyreasons, including but not limited to, requiring the person's input toproceed with an operation (such as installing a program), or providing anotification of an event (such as receipt of a new email) to the person.In the illustrated embodiment, the person is looking at user interfaceelement 802 while user interface element 806 is requesting the person'sattention.

In FIG. 8C, the person has started to look at user interface element806, and thus gaze location 812 has shifted from user interface element802 to user interface element 806. As illustrated, because the gazelocation 812 is now coincident with user interface element 806,electronic device optionally acknowledges user interface element's 806request for attention, and ceases any mechanism user interface element806 was utilizing to request the person's attention (e.g., bouncing,flashing).

FIGS. 8D-8E illustrate another exemplary action that the electronicdevice optionally performs based on the gaze location 812 in userinterface 800. FIG. 8D illustrates user interface 800, as describedabove. In FIG. 8D, gaze location 812 is between the user interfaceelements in user interface 800.

In FIG. 8E, the person has started to look at user interface element806, and thus gaze location 812 is coincident with user interfaceelement 806. As a result, electronic device optionally emphasizes userinterface element 806 relative to other user interface elements. Inother words, the user interface element at which the person is lookingis optionally emphasized relative to other user interface elements. Inthe illustrated example, user interface element 806 is enlarged whilethe sizes of the remaining user interface elements remain constant. Insome implementations, emphasizing a user interface element isaccomplished by increasing its brightness relative to other userinterface elements, promoting its position in a user interface elementz-order, blurring or dimming other user interface elements, changing itsposition relative to other user interface elements, or any othermechanism for emphasizing the user interface element relative to otheruser interface elements.

In some embodiments, the electronic device optionally determines one ormore portions of the display device that are not coincident with thegaze location and a particular user interface element, and adjusts anoutput of the one or more portions of the display device that are notcoincident with the gaze location. In other words, the electronic deviceoptionally dims or turns off parts of the display device that the personis not looking at.

In some embodiments, a user interface element coincident with the gazelocation becomes a selected user interface element. In some embodiments,a particular user interface element includes moving items, such as astock ticker including scrolling stock quotes. The electronic deviceoptionally determines which of the moving items (e.g., stock quotes) thegaze location coincides with, and the electronic device optionallymodifies the movement of the looked-at moving item (e.g., slowing downor stopping the movement) so that the particular moving item is visiblein the user interface. For example, when a person looks at a specificstock quote in a scrolling stock quote list, the electronic deviceoptionally stops the scrolling of the stock quotes so that the looked-atstock quote is visible and can be more easily read by the person.

In some embodiments, the user interface element including the movingitems is an autoscrolling list, and the moving items are elements of theautoscrolling list. In some embodiments, the user interface element isan image catalog (e.g., a collection of photographs, album artwork,etc.), and the moving items are images in the image catalog. In someembodiments, the user interface element is a content bar (e.g., achannel bar, an application bar, etc.), and the moving items arerepresentations of content items accessible form the content bar (e.g.,channels, programs, and apps that are presented to the person inresponse to detecting activation of a corresponding representation inthe content bar).

In some embodiments, a particular user interface element includescontent. For example, the user interface element optionally containstext and/or images. In such a circumstance, when the gaze locationcoincides with the user interface element, the electronic deviceoptionally designates the content in the user interface element as beingread. For example, the electronic device optionally tracks when a personhas read all portions of a contract (or tracks which portions of thecontract the person has read), a message, an email, etc., and when theelectronic device determines that the person has read the entirecontract/message/email, the electronic device optionally designates thecontract/message/email as being read. In some embodiments, theelectronic device designates portions of the contract/message/email asbeing read as they are read by the person.

In some embodiments, the gaze location is used to designate a userinterface element as being an active user interface element. Forexample, if the gaze location is coincident with a particular userinterface element, the electronic device optionally designates thatparticular user interface element as being an active user interfaceelement. Then, when input from an input device (e.g., a mouse) forinteracting with an active user interface element is received by theelectronic device, the electronic device facilitates interaction withthe particular user interface element (e.g., the active user interfaceelement) in accordance with the input. For example, a person optionallydesignates which of a plurality of application windows the persondesires to scroll with a scrolling input on a mouse by, for example,looking at the appropriate application window. The person optionallyalters this designation simply by looking at different applicationwindows. Then, the person optionally provides a scrolling input, forexample, to the electronic device, and the application window at whichthe person is looking optionally scrolls in response to the scrollinginput.

In some embodiments, the input device has an associated cursor (e.g.,mouse pointer) in the user interface. In such circumstances, theelectronic device optionally ignores the location of the cursor indesignating the user interface element with which the gaze location iscoincident as the active user interface element. For example, ascrolling input receiving from a mouse optionally scrolls an applicationwindow the person is looking at, and not the application window overwhich the mouse curser is located.

FIGS. 9A-9C are flow diagrams illustrating a method 900 of performing anaction based in part on determining a gaze location in accordance withsome embodiments (e.g., as described above with reference to FIGS.8A-8E). The method 900 is performed at an electronic device such asdevice 500 as described above with reference to FIG. 5 , including a settop box or other user interface generating device that is incommunication with a remote control and a display device. Someoperations in method 900 are, optionally, combined and/or the order ofsome operations is, optionally, changed.

As described below, the method 900 provides ways in which a deviceoptionally reacts to determining a gaze location of a user. The methodreduces the cognitive burden on a user when interacting with a userinterface on the device, and reduces the need for the user to physicallyinteract with the device, thereby creating a more efficienthuman-machine interface. For battery-operated electronic devices,increasing the efficiency of the user's interaction with the userinterface conserves power and increases the time between batterycharges.

In some embodiments, an electronic device (e.g., a set top box or otheruser interface generating device that is in communication with a remotecontrol and a display device) with one or more processors and memorygenerates (902) a user interface (e.g., user interface 800 in FIG. 8A)for display on a display device (e.g., a television or other displaydevice). In some embodiments, the user interface includes a first userinterface element (904) (e.g., an icon, an application window, a button,etc., such as user interface element 806 in FIG. 8A). In someembodiments, the first user interface element comprises a user interfaceelement requesting attention (906) (e.g., a bouncing icon, such as userinterface element 806 in FIG. 8B). In some embodiments, the userinterface further includes a second user interface element (908) (e.g.,the first user interface element is a first application window and thesecond user interface element is a second application window). In someembodiments, the first user interface element comprises a plurality ofmoving items (910) (e.g., stock ticker with scrolling stock quotes). Insome embodiments, the first user interface element comprises anautoscrolling list, and the plurality of moving items comprise elementsof the autoscrolling list (912). In some embodiments, the first userinterface element comprises an image catalog (e.g., collection ofphotographs, album artwork, etc.), and the plurality of moving itemscomprise images in the image catalog (914). In some embodiments, thefirst user interface element comprises a content bar (e.g., channel bar,app bar), and the plurality of moving items comprise representations ofcontent items accessible from the content bar (916) (e.g., channels,programs, apps that are presented to the user in response to detectingactivation of a corresponding representation in the content bar). Insome embodiments, the first user interface element comprises content(918) (e.g., text, images, etc.).

In some embodiments, the device determines (920) a gaze location in theuser interface (e.g., gaze location 812 in FIG. 8A). In someembodiments, the gaze location comprises a location in the userinterface to which a person's gaze is directed (922) (e.g., the devicedetermines where in the user interface the person is looking; thedetermination is optionally based on identifying the person's eyes andtheir direction).

In some embodiments, the device determines (924) whether the gazelocation in the user interface coincides with the first user interfaceelement (e.g., the device determines whether the person is looking atthe first user interface element, such as whether gaze location 812coincides with user interface element 806 in FIG. 8B).

In accordance with a determination that the gaze location coincides withthe first user interface element (926), in some embodiments, the deviceperforms (928) an action with respect to the first user interfaceelement (e.g., acknowledging a displayed message). In some embodiments,performing the action with respect to the first user interface elementcomprises acknowledging the request for attention in response todetermining that the gaze location coincides with the first userinterface element (930) (e.g., the icon stops bouncing, such asacknowledging user interface element 806′s request for attention in FIG.8C). In some embodiments, performing the action with respect to thefirst user interface element comprises emphasizing the first userinterface element relative to the second user interface element (932)(e.g., emphasizing (brightness, blur, size, etc.) the application windowthat the user is looking at relative to other application window(s) theuser is not looking at, such as is illustrated in FIGS. 8D-8E). In someembodiments, performing the action with respect to the first userinterface element comprises selecting the first user interface element(934) (e.g., select a user interface element the user is looking at). Insome embodiments, performing the action with respect to the first userinterface element comprises designating the content in the first userinterface element as being read (936) (e.g., track whether a user hasread all portions of a contract, message, email, etc.). In someembodiments, performing the action with respect to the first userinterface element comprises designating the first user interface elementas an active user interface element (938).

In some embodiments, the device determines (940) one or more portions ofthe display device that are not coincident with the gaze location andthe first user interface element. In some embodiments, performing theaction with respect to the first user interface element comprisesadjusting an output of the one or more portions of the display devicethat are not coincident with the gaze location (942) (e.g., dimming orturning off parts of the display that the user is not looking at).

In some embodiments, the device determines (944) a first item of theplurality of moving items with which the gaze location coincides. Insome embodiments, performing the action with respect to the first userinterface element comprises modifying a movement (e.g., slowing down orstopping) of the first item of the plurality of moving items such thatthe first item is visible in the user interface (946) (e.g., when theuser looks at a specific stock quote, stopping the scrolling of thestock quotes so that the looked-at stock quote is visible and can bemore easily read by the user).

In some embodiments, the device receives (950) a first input from aninput device (e.g., mouse) for interacting with an active user interfaceelement. In some embodiments, a location of a cursor associated with theinput device is coincident with the second user interface element (952).In some embodiments, designating the first user interface element as theactive user interface element comprises ignoring the location of thecursor in designating the first user interface element as the activeuser interface element (954) (e.g., scrolling the application window theuser is looking at, not the application window over which the cursor islocated). In some embodiments, in response to the first input, thedevice interacts with the first user interface element in accordancewith the first input (956) (e.g., the window that the user is looking atwill scroll in response to a scroll input received on a mouse).

It should be understood that the particular order in which theoperations in FIGS. 9A-9C have been described is merely exemplary and isnot intended to indicate that the described order is the only order inwhich the operations could be performed. One of ordinary skill in theart would recognize various ways to reorder the operations describedherein. Additionally, it should be noted that details of other processesdescribed herein with respect to other methods described herein (e.g.,methods 700, 1100, 1300 and 1500) are also applicable in an analogousmanner to method 900 described above with respect to FIGS. 9A-9C. Forexample, the user interfaces, display device, user interface elementsand gaze location described above with reference to method 900optionally have one or more of the characteristics of the userinterfaces, display devices, user interface elements and gaze locationsdescribed herein with reference to other methods described herein (e.g.,methods 700, 1100, 1300 and 1500). For brevity, these details are notrepeated here.

The operations in the information processing methods described aboveare, optionally implemented by running one or more functional modules inan information processing apparatus such as general purpose processors(e.g., as described above with respect to FIGS. 1A, 3 and 5 ) orapplication specific chips.

The operations described above with reference to FIGS. 9A-9C are,optionally, implemented by components depicted in FIGS. 1A-1B. Forexample, generation operation 902, determination operations 920 and 924,and performing operation 928 are, optionally implemented by event sorter170, event recognizer 180, and event handler 190. Event monitor 171 inevent sorter 170 detects a contact on touch-sensitive display 112, andevent dispatcher module 174 delivers the event information toapplication 136-1. A respective event recognizer 180 of application136-1 compares the event information to respective event definitions186, and determines whether a first contact at a first location on thetouch-sensitive surface corresponds to a predefined event or sub-event,such as selection of an object on a user interface. When a respectivepredefined event or sub-event is detected, event recognizer 180activates an event handler 190 associated with the detection of theevent or sub-event. Event handler 190 optionally utilizes or calls dataupdater 176 or object updater 177 to update the application internalstate 192. In some embodiments, event handler 190 accesses a respectiveGUI updater 178 to update what is displayed by the application.Similarly, it would be clear to a person having ordinary skill in theart how other processes can be implemented based on the componentsdepicted in FIGS. 1A-1B.

Audio-Based Presence Detection

Determining whether a person is present at a device can be beneficial inproviding that person an enhanced user experience with the device. Forexample, if a television were able to determine that a person is in thepresence of the television, the television could forgo going to sleep sothe person's television-viewing can remain uninterrupted. Theembodiments described below describe ways in which a device candetermine whether a person is present at the device. A device that isable to determine whether a person is present at the device reduces theamount of interaction needed between the person and the device, and thusreduces the power usage of the device and increases battery life forbattery-powered devices.

FIG. 10 illustrates exemplary ways in which a device optionallydetermines whether a person is present at the device in accordance withsome embodiments of the disclosure. The embodiments in these figures areused to illustrate the processes described below, including theprocesses described with reference to FIGS. 11A-11B.

FIG. 10 illustrates an exemplary spatial configuration of devicesaccording to embodiments of the disclosure. Device 1001 is optionally adevice such as device 500 as described above with reference to FIG. 5 ,including a set top box or other user interface generating device thatis in communication with a remote control and a display device. In theillustrated embodiment, device 1001 is coupled to sound-playback device1002. Sound-playback device 1002 is optionally a speaker or any devicehaving a speaker, such as a television. In the illustrated example,sound-playback device 1002 has speaker 1004. Sound-playback device 1002optionally corresponds to one or more of multifunction device 100 inFIGS. 1A and 2 , device 300 in FIG. 3 , and display 514 in FIG. 5 ;speaker 1004 optionally corresponds to speaker 111 in FIGS. 1A and 2 .Device 1008 is a sound-receiving device. Sound-receiving device 1008 isoptionally a microphone or any device having a microphone, such as amobile and/or wearable device. Sound-receiving device optionallycorresponds to one or more of multifunction device 100 in FIGS. 1A and 2, microphone 113 in FIGS. 1A and 2 , and device 300 in FIG. 3 .

In some embodiments, device 1001 optionally determines whethersound-receiving device 1008 is present at sound-playback device 1002 bygenerating a first sound 1006 for playback on the sound-playback device.The first sound 1006 is optionally an audible sound, or an inaudiblesound that is outside of the normal range of hearing for a human (e.g.,below 20 Hz or above 20,000 Hz). While or after sound-playback device1002 has played the first sound 1006, device 1001 optionally receives anoutput from the microphone on sound-receiving device 1008. Device 1001optionally receives this output via a wireless connection withsound-receiving device 1008, such as via Bluetooth Low Energy (sometimescalled Bluetooth LE).

Device 1001 then optionally determines whether sound-receiving device1008 is present at sound-playback device 1002 based on a plurality ofpresence criteria. If the presence criteria are satisfied, device 1001optionally designates a user of the sound-receiving device 1008 as beingpresent at sound-playback device 1002. For example, when the presencecriteria are satisfied with respect to a mobile phone, a user of themobile phone is determined to be present at the sound-playback device.In some embodiments, only sound-receiving device 1008, and not a user ofthe sound-receiving device, is designated as being present when thepresence criteria are satisfied. If the presence criteria are notsatisfied with respect to the mobile phone, the user of the mobile phoneis determined to be not present at the sound-playback device—thisdetermination optionally encompasses simply not designating the user aspresent, or changing a “present” determination for that user to a “notpresent” determination.

In some embodiments, the presence criteria include sound-detectioncriteria that is satisfied when the output from the microphone on thesound-receiving device 1008 corresponds to the first sound 1006 forplayback on sound-playback device 1002. For example, device 1001optionally determines whether sound-receiving device 1008 detected thefirst sound 1006 played by the sound-playback device 1002 by comparingthe first sound with the output from the microphone on thesound-receiving device. If the output from the microphone and the firstsound 1006 match, then in some embodiments, the sound-detection criteriais satisfied.

In some embodiments, the presence criteria also include verificationcriteria that is satisfied when presence of the sound-receiving device1008 at the sound-playback device 1002 is verified using something otherthan the first sound 1006. In other words, the verification criteria isoptionally satisfied when device 1001 verifies that sound-receivingdevice 1008 is present at sound-playback device 1002 in a way other thanthrough detection of first sound 1006. Some such verification methodsoptionally include facial recognition and/or sound-receiving device 1008motion checks, as will be described in more detail below. Thus, in someembodiments, the user of the sound-receiving device 1008 is notdesignated as being present at sound-playback device 1002 until both thesound-detection criteria and the verification criteria are satisfied.

In some embodiments, designating the user of sound-receiving device 1008as being present at sound-playback device 1002 includes determining auser associated with the sound-receiving device (e.g., determining whouses, owns, or is registered on the sound-receiving device), anddesignating that user as being present at the sound-playback device.

In some embodiments, prior to generating first sound 1006 for playbackon sound-playback device 1002, device 1001 optionally transmits a wakecommand to sound-receiving device 1008 to request that thesound-receiving device turn on its microphone. For example, device 1001optionally sends a wireless signal to a mobile device to have the mobiledevice turn on its microphone. In this way, sound-receiving device 1008need not always keep its microphone on, preserving battery life.

In some embodiments, the verification criteria include a criterion thatis satisfied when a face of the user of the sound-receiving device 1008is recognized at the sound-playback device 1002. In such cases, device1001 optionally ensures via facial recognition that the relevant user isat sound-playback device 1002 before designating that user as beingpresent. In some embodiments, sound-playback device 1002 has a camera(such as an RGB, IR and/or depth camera) from which device 1001optionally analyzes one or more images to check for facial recognitionof the user.

In some embodiments, the verification criteria include a criterion thatis satisfied when sound-receiving device 1008 has sensed motion during amost recent time period (e.g., a time period such as the last 1, 5, 15,or 30 minutes). In such cases, device 1001 optionally designates theuser as being present at sound-playback device 1002 only ifsound-playback device 1008 has recently sensed motion. Device 1001 doesthis so as to not designate a user as being present when that user hasleft, for example, their telephone in the room with sound-playbackdevice 1002, and the user is not actually there.

In some embodiments, the verification criteria include a criterion thatis satisfied when sound-receiving device 1008 is on the same wirelessnetwork (e.g., WiFi network or any wireless communication network) asthe sound-playback device. In other words, in such embodiments, device1001 checks to make sure that sound-playback device 1002 andsound-receiving device 1008 are on the same wireless network beforedesignating the user as being present at the sound-playback device.

In some embodiments, the above presence determination is optionallyre-performed when sound-receiving device 1008 has sensed a large amountof motion, to determine whether the large amount of motion correspondsto the user having left the room with the sound-receiving device whilepreviously having been in the room, for example. In such embodiments,device 1001 optionally detects motion of sound-receiving device 1008. Inaccordance with a determination that the motion exceeds a predefinedthreshold (e.g., more than a predefined change in location, more than apredefined change in velocity, more than a predefined change inacceleration), the device optionally re-evaluates the plurality ofpresence criteria as described above to determine whethersound-receiving device 1008 remains at the sound-playback device 1002.

In some embodiments, sound-playback device 1002 is configured to gointo, for example, a sleep state after a certain amount of time haspassed without any activity being sensed at the sound-playback device.For example, a television is optionally configured to go to sleep after30 minutes have passed without any input from a remote control or anyvideo input to the television. In such cases, device 1001 optionallyforgoes transitioning sound-playback device 1002 into the sleep statewhen the presence criteria are satisfied. For example, sound-playbackdevice 1002 is optionally configured to transition from a first state(e.g., active or awake state, and optionally also includes a person'sstatus on a chat program, for example) to a second state (e.g., inactiveor sleep state, also optionally including a person's status on the chatprogram, for example) in response to one or more transition criteria(e.g., lack of activity in the user interface of the sound-playbackdevice for longer than a threshold time period) being satisfied at thesound-playback device (e.g., transitioning a person's chat programstatus from “active” to “inactive” after a threshold time period ofinactivity). However, in accordance with the determination that theplurality of presence criteria are satisfied, device 1001 optionallyforgoes transitioning from the first state of the sound-playback deviceto the second state of the sound-playback device in response to the oneor more transition criteria being satisfied at the sound-playbackdevice. In other words, despite the one or more transition criteriabeing met, device 1001 optionally maintains the sound-playback device1002 in the first (e.g., active or awake) state due to the determinedpresence of the person at the sound-playback device.

FIGS. 11A-11B are flow diagrams illustrating a method 1100 of performingpresence detection in accordance with some embodiments (e.g., asdescribed above with reference to FIG. 10 ). The method 1100 isperformed at an electronic device such as device 500 as described abovewith reference to FIG. 5 , including a set top box or other userinterface generating device that is in communication with a remotecontrol and a display device. Some operations in method 1100 are,optionally, combined and/or the order of some operations is, optionally,changed.

As described below, the method 1100 provides ways in which a deviceoptionally determines the presence of a user without a user needing tomanually designate his/her presence. The method reduces the cognitiveburden on a user when interacting with the device, and reduces the needfor the user to physically interact with the device, thereby creating amore efficient human-machine interface. For battery-operated electronicdevices, increasing the efficiency of the user's interaction with thedevice conserves power and increases the time between battery charges.

In some embodiments, an electronic device (e.g., a set top box or otheruser interface generating device that is in communication with a remotecontrol and a display device, such as device 1001 in FIG. 10 ) with oneor more processors and memory generates (1102) a first sound (e.g., anaudible or inaudible sound that is outside of the normal range ofhearing for a human (below 20 Hz or above 20,000 Hz), such as firstsound 1006 in FIG. 10 ) for playback on a sound-playback device (e.g., aspeaker or any device having a speaker, such as a television and/ordevice 1002 in FIG. 10 ). In some embodiments, the sound-playback deviceis configured to transition from a first state (e.g., active or awakestate, and optionally also includes a person's status on a chat program,for example) to a second state (e.g., inactive or asleep state, alsooptionally including a person's status on the chat program, for example)in response to one or more transition criteria (e.g., lack of activityin the user interface for longer than a threshold time period) beingsatisfied at the sound-playback device (1104) (e.g., transitioning aperson's chat program status from “active” to “inactive” after athreshold time period of inactivity).

In some embodiments, the device receives (1106) (e.g., via wirelesscommunication, such as Bluetooth LE) an output from a microphone on asound-receiving device (e.g., a microphone or any device having amicrophone, such as a mobile and/or wearable device, sending dataresulting from detecting the first sound to the electronic device; forexample, device 1008 in FIG. 10 ). In some embodiments, prior togenerating the first sound for playback on the sound-playback device,the device transmits a wake command to the sound-receiving device forturning on the microphone on the sound-receiving device (1108) (e.g.,the device sends a wireless signal to a mobile device to have the mobiledevice turn on its microphone).

In some embodiments, the device determines (1110) whether thesound-receiving device is present at the sound-playback device based ona plurality of presence criteria, including sound-detection criteriathat is satisfied when the output from the microphone on thesound-receiving device corresponds to the first sound for playback onthe sound-playback device (1112) (e.g., the device determines whetherthe sound-receiving device detected the first sound played by thesound-playback device by comparing the first sound with the output fromthe microphone on the sound-receiving device), and verification criteriathat is satisfied when a presence of the sound-receiving device at thesound-playback device is verified using a quantity other than the firstsound (1114) (e.g., by facial recognition, motion checks, etc.). In someembodiments, the verification criteria include a criterion that issatisfied when a face of the user of the sound-receiving device isrecognized at the sound-playback device (1116) (e.g., the device looksfor a facial recognition match and a sound-based presence determinationbefore designating the user as present). In some embodiments, theverification criteria include a criterion that is satisfied when thesound-receiving device has sensed motion during a most recent timeperiod (1118) (e.g., a time period such as the last 1, 5, 15, or 30minutes so as to ignore the sound-based presence determination if themobile device has been left on a table in the room somewhere). In someembodiments, the verification criteria include a criterion that issatisfied when the sound-receiving device resides on a same wirelessnetwork as the sound-playback device (1120) (e.g., the device checks forsound-based presence and whether the sound-playback device and thesound-receiving device are on the same wireless network connectionbefore designating the user as being present).

In accordance with a determination that the plurality of presencecriteria are satisfied, in some embodiments, the device designates(1122) a user of the sound-receiving device as being present at thesound-playback device (e.g., when the presence criteria are satisfiedwith respect to a mobile phone, a user of the mobile phone is determinedto be present at the sound-playback device. If the presence criteria arenot satisfied with respect to the mobile phone, the user of the mobilephone is determined to be not present at the sound-playback device—thisdetermination optionally encompasses simply not designating the user aspresent, or changing a “present” determination for that user to a “notpresent” determination). In some embodiments, designating the user ofthe sound-receiving device as being present at the sound-playback devicecomprises determining a user associated with the sound-receiving device(e.g., determining who uses, owns, or is registered on thesound-receiving device), and designating the user as being present atthe sound-playback device (1124). In some embodiments, in accordancewith the determination that the plurality of presence criteria aresatisfied, the device forgoes (1126) transitioning from the first stateof the sound-playback device to the second state of the sound-playbackdevice in response to the one or more transition criteria beingsatisfied at the sound-playback device (e.g., despite the one or moretransition criteria being met, maintaining the sound-playback device inthe first (e.g., active or awake) state due to the presence of theperson in front of the display device).

In some embodiments, the device detects (1130) motion of thesound-receiving device. In some embodiments, in accordance with adetermination that the motion exceeds a predefined threshold (e.g., morethan a predefined change in location, more than a predefined change invelocity, more than a predefined change in acceleration), the devicere-evaluates (1132) the plurality of presence criteria to determinewhether the sound-receiving device remains at the sound-playback device(e.g., the device determines whether a user has left the room whilepreviously having been in the room).

It should be understood that the particular order in which theoperations in FIGS. 11A-11B have been described is merely exemplary andis not intended to indicate that the described order is the only orderin which the operations could be performed. One of ordinary skill in theart would recognize various ways to reorder the operations describedherein. Additionally, it should be noted that details of other processesdescribed herein with respect to other methods described herein (e.g.,methods 700, 900, 1300 and 1500) are also applicable in an analogousmanner to method 1100 described above with respect to FIGS. 11A-11B. Forbrevity, these details are not repeated here.

The operations in the information processing methods described aboveare, optionally implemented by running one or more functional modules inan information processing apparatus such as general purpose processors(e.g., as described above with respect to FIGS. 1A, 3 and 5 ) orapplication specific chips.

The operations described above with reference to FIGS. 11A-11B are,optionally, implemented by components depicted in FIGS. 1A-1B. Forexample, generation operation 1102, receiving operation 1106,determination operation 1110 and designation operation 1122 are,optionally implemented by event sorter 170, event recognizer 180, andevent handler 190. Event monitor 171 in event sorter 170 detects acontact on touch-sensitive display 112, and event dispatcher module 174delivers the event information to application 136-1. A respective eventrecognizer 180 of application 136-1 compares the event information torespective event definitions 186, and determines whether a first contactat a first location on the touch-sensitive surface corresponds to apredefined event or sub-event, such as selection of an object on a userinterface. When a respective predefined event or sub-event is detected,event recognizer 180 activates an event handler 190 associated with thedetection of the event or sub-event. Event handler 190 optionallyutilizes or calls data updater 176 or object updater 177 to update theapplication internal state 192. In some embodiments, event handler 190accesses a respective GUI updater 178 to update what is displayed by theapplication. Similarly, it would be clear to a person having ordinaryskill in the art how other processes can be implemented based on thecomponents depicted in FIGS. 1A-1B.

Distance Determination

Displaying different user interfaces on a device based on whether aperson is far away from the device or close to the device can enhancethe person's interaction with the device. For example, a television canpresent a user interface with large user interface elements that providesummary information (e.g., information about the current program and thenext program) when a person is far away from the television, and canpresent more detailed information (e.g., a program guide withinformation about programs from multiple channels and multiple timeslots) when the person is close to the television. The embodimentsdescribed below describe ways in which a person's interaction with adevice can be enhanced by providing different user interfaces atdifferent distances. Enhancing a person's interactions with a devicereduces the amount of time needed by the person to perform operations,and thus reduces the power usage of the device and increases batterylife for battery-powered devices.

FIGS. 12A-12B illustrate exemplary ways in which different userinterfaces are optionally presented at different distances in accordancewith some embodiments of the disclosure. The embodiments in thesefigures are used to illustrate the processes described below, includingthe processes described with reference to FIGS. 13A-13D.

FIG. 12A illustrates an exemplary user interface 1200 for display ondisplay device 1002 when person 1210 is far from the display device.Display device 1002 is optionally any display device, such as atelevision or other display device. Display device 1002 is optionallycoupled to device 1001, which is optionally a device such as device 500as described above with reference to FIG. 5 , including a set top box orother user interface generating device that is in communication with aremote control and a display device.

In some embodiments, device 1001 determines whether person 1210 isfurther than threshold distance 1208 from display device 1002, or closerthan the threshold distance from the display device. Device 1001optionally makes this distance determination based on informationreceived from one or more optical sensors, such as IR, depth and/or RGBcameras. In the illustrated embodiment, person 1210 is further thanthreshold distance 1208 from display device 1002. Threshold distance1208 is optionally any number of distances, fixed or variable, such as1/10, ½, 1, 3, or 5 meters from display device 1002.

If, as illustrated in FIG. 12A, device 1001 determines that person 1210is further than threshold distance 1208 from display device 1002, thedevice optionally provides user interface 1200 for display on thedisplay device. User interface 1200 is optionally any user interfacesuited for or intended for presentation when person 1210 is further thanthreshold distance 1208 from display device 1002. For example, userinterface 1200 is optionally a user interface including summaryinformation, a screen saver, application windows without relativelysmall user interface elements, user interface elements at a relativelylarge size, and/or status information (e.g., date, time, weather, newsheadlines, stock prices, number of unread emails, etc.). In theillustrated embodiment, user interface 1200 includes two user interfaceelements 1202 and 1204. User interface element 1202 is optionally acontent item, such as a video or an image. User interface element 1204is optionally an information bar, providing information about thecontent item (e.g., user interface element 1202), such as a title and adescription.

FIG. 12B illustrates an exemplary user interface 1201 for display ondisplay device 1002 when person 1210 is close to the display device. Inthe illustrated embodiment, device 1001 has determined that person 1210is closer than threshold distance 1208 from display device 1002. Thus,device 1001 optionally provides user interface 1201 for display ondisplay device 1002. User interface 1201 is optionally any userinterface suited for or intended for presentation when person 1210 iscloser than threshold distance 1208 from display device 1002. Forexample, user interface 1201 is optionally a user interface includingdetailed information, a home screen, application windows includingrelatively small user interface elements, and/or user interface elementsat a relatively small size. In the illustrated embodiment, userinterface includes three user interface elements 1202, 1204, and 1206.User interface elements 1202 and 1204 are optionally smaller versions ofthe same user interface elements that were in user interface 1200 (e.g.,the user interface that is displayed when the person is far away fromthe display device). User interface element 1206 is optionally a userinterface element that is not included in user interface 1200 (e.g., theuser interface that is displayed when the person is far away from thedisplay device), and is only included in user interface 1201. Forexample, user interface element 1206 is optionally a channel orapplication bar from which various content (such as media content orapplications) is optionally accessed. User interface element 1206 isoptionally only included in user interface 1201 because it is optionallybetter suited for close viewing/interaction as opposed to farviewing/interaction.

As described above, device 1001 optionally provides a different userinterface for display on display device 1002 depending on whether person1210 is further than or closer than threshold distance 1208 from displaydevice 1002.

In some embodiments, the device optionally identifies a face of theperson based on the information received from the one or more opticalsensors. In such implementations, the determination as to whether theperson is further than the threshold distance from the display device orcloser than the threshold distance from the display device is optionallybased on the size of the face of the person (e.g., by comparing the sizeof face to a “standard” sized face known to be at a reference distance,and extrapolating the person's distance based on the comparison).

In some embodiments, the device optionally detects movement of theperson from a distance further than the threshold distance to a distancecloser than the threshold distance from the display device. In responseto detecting the movement, the device optionally provides the “close”user interface (e.g., the user interface that is displayed when theperson is near to the display device) for display on the display device.

In some embodiments, the device optionally detects movement of theperson from a distance closer than the threshold distance to a distancefurther than the threshold distance from the display device. In responseto detecting the movement, the device optionally provides the “far” userinterface (e.g., the user interface that is displayed when the person isfar from the display device) for display on the display device.

In some embodiments, the person is further than the threshold distancefrom the display device, and the “far” user interface is provided fordisplay on the display device. The “far” user interface optionallyincludes a first user interface element (e.g., an email graphic) and asecond user interface element (e.g., a music collection graphic). Whilethe “far” user interface is displayed on the display device, the deviceoptionally determines a gaze location in the “far” user interface, wherethe gaze location is a location in the “far” user interface to which theperson looking. The device optionally detects movement of the person toa distance closer than the threshold distance from the display device(e.g., the person is moving towards the display device). In response todetecting the movement of the person, the device provides the “close”user interface (e.g., the user interface that is displayed when theperson is near the display device) for display on the display device,where the “close” user interface includes information based on the gazelocation. For example, when a person is looking at an email, when theperson moves closer to the display, additional details about the emailare optionally displayed (e.g., a sender, other recipients, timestamps,etc.). As another example, if the person is looking at an image of coverart for a music album, when the person moves closer to the display,additional details about the album are displayed (e.g., a track list,artist, track times, etc.). Optionally, when the additional informationis displayed based on the gaze location, additional information is notdisplayed for other user interface elements that the person is notlooking at (e.g., when cover art for multiple albums is displayed, atrack list is only displayed for the album corresponding to the coverart that the person is looking at when the person gets closer to thedisplay and is not displayed for other albums).

In some embodiments, the device logs the person out of any areas theperson might be logged into when the person moves away from the displaydevice. For example, the person might be close to the display device,and logged into one or more areas on the device (e.g., iTunes store,email service, etc.). The device optionally detects movement of theperson to a distance further than the threshold distance from thedisplay device (e.g., the person is moving away from the displaydevice), and in response, optionally logs out of the one or more areason the electronic device. For example, the device optionally logs out ofthe person's iTunes store account when that person leaves the displaydevice.

In some embodiments, the “far” and “close” user interfaces include thesame user interface element at different sizes. For example, the “far”user interface optionally includes a user interface element (e.g., textstring, application window, button, etc.) at a first size. The “close”user interface optionally includes the same user interface element at asecond size, smaller than the first size.

In some embodiments, the “far” user interface provides summaryinformation about something, and the “close” user interface providesdetailed information about that same thing. For example, the “far” userinterface optionally includes summary information about a first quantity(e.g., weather, unread emails, etc.), and the “close” user interfaceoptionally includes detailed information about the first quantity. Thus,in some embodiments, the “far” user interface optionally includes asnapshot of the weather, such as a high temperature, low temperature,and current temperature for the current day, and the “close” userinterface optionally includes more detailed information about theweather, such as the above temperatures as well as the humidity, chanceof rain, wind, and wind chill for the current day.

In some embodiments, the “far” user interface does not include one ormore user interface elements that are optionally included in the “close”user interface. For example, the “close” user interface optionallyincludes application windows, a menu bar and an application dock.Because the menu bar and application bar could be difficult to viewand/or interact with at a distance, the “far” user interface optionallyincludes only the application windows, and not the menu bar or theapplication dock.

FIGS. 13A-13D are flow diagrams illustrating a method 1300 of providingdifferent user interfaces at different distances in accordance with someembodiments (e.g., as described above with reference to FIGS. 12A-12B).The method 1300 is performed at an electronic device such as device 500as described above with reference to FIG. 5 , including a set top box orother user interface generating device that is in communication with aremote control and a display device. Some operations in method 1300 are,optionally, combined and/or the order of some operations is, optionally,changed.

As described below, the method 1300 provides ways in which a deviceoptionally provides different user interfaces based on the distance of auser from a display device. The method reduces the cognitive burden on auser when interacting with a user interface on the device, and reducesthe need for the user to physically interact with the device, therebycreating a more efficient human-machine interface. For battery-operatedelectronic devices, increasing the efficiency of the user's interactionwith the user interface conserves power and increases the time betweenbattery charges.

In some embodiments, an electronic device (e.g., a set top box or otheruser interface generating device that is in communication with a remotecontrol and a display device, such as device 1001 in FIG. 12A) with oneor more processors and memory determines (1302) whether a person (e.g.,person 1210 in FIG. 12A) is further than a threshold distance (e.g.,further than a distance such as 1/10, ½, 1, 3 or 5 meters, such asdistance 1208 in FIG. 12A) from a display device (e.g., a television orother display device, such as device 1002 in FIG. 12A) or closer thanthe threshold distance from the display device. In some embodiments,determining whether the person is further than the threshold distancefrom the display device or closer than the threshold distance from thedisplay device is based on information received from one or more opticalsensors (1304) (e.g., IR, depth, and/or RGB cameras).

In accordance with a determination that the person is further than thethreshold distance from the display device, in some embodiments, thedevice provides (1306) a first user interface (e.g., a user interfaceincluding summary information, a screen saver, application windowswithout small user interface elements, user interface elements at alarger size, status information (e.g., date, time, weather, newsheadlines, stock prices, number of unread emails), such as userinterface 1200 in FIG. 12A) for display on the display device. In someembodiments, the first user interface (e.g., the user interface that isdisplayed when the user is far away from the display device) includes auser interface element (e.g., text string, application window, button,etc.) at a first size (1308) (e.g., user interface element 1202 in FIG.12A). In some embodiments, the first user interface includes summaryinformation about a first quantity (1310) (e.g., weather, unread emails,etc.). In some embodiments, the first user interface includes a firstset of user interface elements (1312) (e.g., application windows, suchas user interface elements 1202 and 1204 in FIG. 12A).

In accordance with a determination that the person is closer than thethreshold distance from the display device, in some embodiments, thedevice provides (1314) a second user interface (e.g., a user interfaceincluding detailed information, a home screen, application windowsincluding small user interface elements, user interface elements at asmaller size, such as user interface 1201 in FIG. 12B) for display onthe display device. In some embodiments, the second user interface isdifferent than the first user interface (1316). In some embodiments, thesecond user interface (e.g., the user interface that is displayed whenthe user is near to the display device) includes the user interfaceelement (e.g., text string, application window, button, etc.) at asecond size, smaller than the first size (1318) (e.g., user interfaceelement 1202 in FIG. 12B). In some embodiments, the second userinterface includes detailed information about the first quantity (1320)(e.g., the first user interface includes a snapshot of the weather, suchas a high temperature, low temperature, and current temperature for thecurrent day, and the second user interface includes more detailedinformation about the weather, such as the above temperatures as well asthe humidity, chance of rain, wind, and wind chill for the current day).In some embodiments, the second user interface includes the first set ofuser interface elements (e.g., the application windows that weredisplayed when the user was far away from the device, such as userinterface elements 1202 and 1204 in FIGS. 12A and 12B) and a second setof user interface elements (1322) (e.g., a relatively small menu bar anddock, such as user interface element 1206 in FIG. 12B, and theapplication windows that were displayed when the user was far away fromthe device).

In some embodiments, the device identifies (1324) a face of the personbased on the information received from the one or more optical sensors.In some embodiments, determining whether the person is further than thethreshold distance from the display device or closer than the thresholddistance from the display device is based on a size of the face of theperson (1326) (e.g., the device compares the size of the face to a“standard” sized face known to be at a reference distance, andextrapolates the person's distance based on the comparison).

In some embodiments, the person is determined to be further than thethreshold distance (e.g., further than a distance such as 1/10, ½, 1, 3or 5 meters) from the display device (1328). In some embodiments, whilethe first user interface (e.g., the user interface that is displayedwhen the user is far away from the display device) is displayed on thedisplay device, the device detects (1330) movement of the person to adistance closer than the threshold distance from the display device. Insome embodiments, in response to detecting the movement of the person tothe distance closer than the threshold distance, the device provides(1332) the second user interface (e.g., the user interface that isdisplayed when the user is near to the display device) for display onthe display device.

In some embodiments, the person is determined to be closer than thethreshold distance (e.g., closer than a distance such as 1/10, ½, 1, 3or 5 meters) from the display device (1334). In some embodiments, whilethe second user interface (e.g., the user interface that is displayedwhen the user is near to the display device) is displayed on the displaydevice, the device detects (1336) movement of the person to a distancefurther than the threshold distance from the display device. In someembodiments, in response to detecting the movement of the person to thedistance further than the threshold distance, the device provides (1338)the first user interface (e.g., the user interface that is displayedwhen the user is far away from the display device) for display on thedisplay device.

In some embodiments, the person is further than the threshold distance(e.g., further than a distance such as 1/10, ½, 1, 3 or 5 meters) fromthe display device, and the first user interface (e.g., the userinterface that is displayed when the user is far away from the displaydevice) is provided for display on the displace display device, thefirst user interface including a first user interface element (e.g., anemail graphic) and a second user interface element (1340) (e.g., a musiccollection graphic). In some embodiments, while the first user interfaceis displayed on the display (1342) the device determines (1344) a gazelocation in the first user interface, wherein the gaze locationcomprises a location in the first user interface to which the person'sgaze is directed, and detects (1346) movement of the person to adistance closer than the threshold distance from the display device(e.g., the person is moving towards the display device). In someembodiments, in response to detecting the movement of the person, thedevice provides (1348) the second user interface (e.g., the userinterface that is displayed when the user is near to the display device)for display on the display device, the second user interface includinginformation based on the gaze location (e.g., based on where in the userinterface the person is looking). For example, when a person is lookingat an email, when the person moves closer to the display, additionaldetails about the email are displayed (e.g., a sender, other recipients,timestamps, etc.). As another example, if the person is looking at animage of cover art for a music album, when the person moves closer tothe display, additional details about the album are displayed (e.g., atrack list, artist, track times, etc.). Optionally, when the additionalinformation is displayed based on the gaze location, additionalinformation is not displayed for other user interface elements that theperson is not looking at (e.g., when cover art for multiple albums isdisplayed, a track list is only displayed for the album corresponding tothe cover art that the person is looking at when the person gets closerto the display and is not displayed for other albums).

In some embodiments, the device determines (1350) whether the gazelocation coincides with the first user interface element or the seconduser interface element in the first user interface (e.g., the devicedetermines whether the person is looking at the email graphic or themusic collection graphic). In some embodiments, in response to detectingthe movement of the person to the distance closer than the thresholddistance from the display device (1352) (e.g., the person is movingtowards the display device), in accordance with a determination that thegaze location coincides with the first user interface element (e.g., theperson is looking at the email graphic), the device provides (1354) thesecond user interface (e.g., the user interface that is displayed whenthe user is near to the display device) for display on the displaydevice, the second user interface including information associated withthe first user interface element (e.g., more information about theperson's emails, such as email previews, names of senders of unreademails, etc.), and in accordance with a determination that the gazelocation coincides with the second user interface element (e.g., theperson is looking at the music collection graphic), the device provides(1356) the second user interface (e.g., the user interface that isdisplayed when the user is near to the display device) for display onthe display device, the second user interface including informationassociated with the second user interface element (e.g., moreinformation about the person's music collection, such as new songs,artists of new songs, etc.).

In some embodiments, the person is closer than the threshold distancefrom the display device, and the person is logged into one or more areas(e.g., iTunes store, email service, etc.) on the electronic device(1358). In some embodiments, the device detects (1360) movement of theperson to a distance further than the threshold distance from thedisplay device (e.g., the person is moving away from the displaydevice). In some embodiments, in response to detecting the movement ofthe person to the distance further than the threshold distance from thedisplay device, the device logs out (1362) of the one or more areas onthe electronic device (e.g., the device logs out of the person's iTunesstore account when they leave the display device).

It should be understood that the particular order in which theoperations in FIGS. 13A-13D have been described is merely exemplary andis not intended to indicate intended to indicate that the describedorder is the only order in which the operations could be performed. Oneof ordinary skill in the art would recognize various ways to reorder theoperations described herein. Additionally, it should be noted thatdetails of other processes described herein with respect to othermethods described herein (e.g., methods 700, 900, 1100 and 1500) arealso applicable in an analogous manner to method 1300 described abovewith respect to FIGS. 13A-13D. For example, the user interfaces, displaydevice, user interface elements, gaze locations and optical sensorsdescribed above with reference to method 1300 optionally have one ormore of the characteristics of the user interfaces, display devices,user interface elements, gaze locations and optical sensors describedherein with reference to other methods described herein (e.g., methods700, 900, 1100 and 1500). For brevity, these details are not repeatedhere.

The operations in the information processing methods described aboveare, optionally implemented by running one or more functional modules inan information processing apparatus such as general purpose processors(e.g., as described above with respect to FIGS. 1A, 3 and 5 ) orapplication specific chips.

The operations described above with reference to FIGS. 13A-13D are,optionally, implemented by components depicted in FIGS. 1A-1B. Forexample, determination operation 1302, and providing operations 1306 and1314 are, optionally implemented by event sorter 170, event recognizer180, and event handler 190. Event monitor 171 in event sorter 170detects a contact on touch-sensitive display 112, and event dispatchermodule 174 delivers the event information to application 136-1. Arespective event recognizer 180 of application 136-1 compares the eventinformation to respective event definitions 186, and determines whethera first contact at a first location on the touch-sensitive surfacecorresponds to a predefined event or sub-event, such as selection of anobject on a user interface. When a respective predefined event orsub-event is detected, event recognizer 180 activates an event handler190 associated with the detection of the event or sub-event. Eventhandler 190 optionally utilizes or calls data updater 176 or objectupdater 177 to update the application internal state 192. In someembodiments, event handler 190 accesses a respective GUI updater 178 toupdate what is displayed by the application. Similarly, it would beclear to a person having ordinary skill in the art how other processescan be implemented based on the components depicted in FIGS. 1A-1B.

Presence Determination

Performing various actions at a device based on whether a person ispresent at the device can be beneficial in providing that person anenhanced user experience with the device. For example, if a televisionwere able to determine whether a person is in the presence of thetelevision, the television could take certain actions that can enhancethe person's experience with the device. The embodiments described belowdescribe ways in which a device can enhance a person's experience withthe device based on whether the person is present at the device.Enhancing a person's experience with a device reduces the amount of timeneeded by the person to perform operations, and thus reduces the powerusage of the device and increases battery life for battery-powereddevices.

FIGS. 14A-14B illustrate exemplary ways in which a person's experiencewith a device is optionally enhanced based on presence detection inaccordance with some embodiments of the disclosure. The embodiments inthese figures are used to illustrate the processes described below,including the processes described with reference to FIGS. 15A-15C.

FIG. 14A illustrates an exemplary user interface 1400 in accordance withembodiments of the disclosure. In the illustrated embodiment, userinterface 1400 includes a video of a ball 1402 moving across the screen.In some embodiments, user interface 1400 includes the playing of anymedia content, such as a movie or music. User interface is optionallyfor display on display device 1002, which is optionally any type ofdisplay device, such as a television. Display device 1002 is optionallycoupled to device 1001. Device 1001 is optionally a device such asdevice 500 as described above with reference to FIG. 5 , including a settop box or other user interface generating device that is incommunication with a remote control and a display device.

In some embodiments, while the media content is playing in userinterface 1400, device 1001 optionally determines whether one or morepresence criteria are satisfied. These presence criteria optionallyinclude a criterion that is satisfied while a person is present atdisplay device 1002. In other words, device 1001 optionally determineswhether a person is present at display device 1002. If device 1001determines that a person is present at display device 1002, the deviceoptionally maintains the playing of the media content in user interface1400, as illustrated. More generally, in some embodiments, if device1001 determines that the one or more presence criteria are satisfied,the device optionally maintains the playing of the media content in userinterface 1400. Whether the one or more presence criteria are satisfiedis optionally determined based on information received from one or moreoptical sensors (e.g., IR, depth, and/or RGB cameras).

FIG. 14B illustrates user interface 1400 when device 1001 determinesthat the one or more presence criteria are not satisfied. In theillustrated embodiment, device 1001 has determined that a person is not(or no longer) present at display device 1002. As a result, device 1001optionally modifies the playing of the media content in a first manner.In the illustrated embodiment, device 1001 has paused the playing of thevideo of the ball 1402 moving across the screen. In some embodiments, inaccordance with a determination that the one or more presence criteriaare not satisfied, device 1001 pauses the media content (asillustrated), bookmarks a current position in the media content, reducesa volume of the media content, or performs any other modification of theplaying of the media content. As such, modification of the playing ofmedia content is based on whether one or more presence criteria aresatisfied.

In some embodiments, the device optionally transfers the playing of themedia content to another device when the one or more presence criteriaare not satisfied. For example, in accordance with a determination thatthe one or more presence criteria are not satisfied, the deviceoptionally determines a second device (e.g., mobile device) associatedwith a first person (e.g., the first person's iPhone). Then, the deviceoptionally transfers the playing of the media content to the seconddevice. For example, the device optionally transfers the playing of themedia content from the display device to the first person's iPhone inresponse to determining that the first person is no longer present atthe display device. This transfer is optionally accomplished, forexample, by transferring an indication of progress of the first personin the media content so that the media content will start playing on thesecond device where the user left off on the first device. In someembodiments, transferring the playing of the media content includesinitiating playback of the media content on the second device. In someembodiments, transferring the playing of the media content includestransferring the media content from the first device to the seconddevice. In some embodiments, the playing of the media content istransferred to the second device without transferring the media contentto the second device (e.g., because the content is already present onboth devices or because both devices are streaming the content from aremote source such as a video or audio streaming service).

In some embodiments, the device modifies the playing of the mediacontent even if the one or more presence criteria are satisfied. Forexample, in some embodiments, in accordance with a determination thatthe one or more presence criteria are satisfied (e.g., the first personis present at the display device), the device optionally determines arelative location of the first person with respect to the displaydevice. In other words, the device optionally determines where inrelation to the display device the first person is. Then, the deviceoptionally modifies the playing of the media content in a second manner(e.g., by beamforming the audio of the media content), different fromthe first manner, based on the relative location of the first person.Thus, in some embodiments, the device optionally adjusts the audiooutput of the media content to beamform the audio to the first person'sposition.

In some embodiments, in accordance with a determination that the one ormore presence criteria are satisfied (e.g., the first person is presentat the display device), the device optionally determines a distance ofthe first person from the display device, and modifies the playing ofthe media content in a second manner, different from the first manner,based on the distance of the first person from the display device. Forexample, in some embodiments, the device adjusts the volume of the mediacontent or the size of the media content in the display based on thedistance of the first person from the display device (e.g., if the firstperson is far from the display device, the device optionally increasedthe volume of the media content).

In some embodiments, the one or more presence criteria include acriterion that is satisfied when the first person is oriented towardsthe display device (e.g., the first person's face, shoulders, and/ortorso are positioned towards the display device). In some embodiments,the one or more presence criteria include a criterion that is satisfiedwhen the first person's gaze is directed at the display device (e.g.,the first person is looking at the display device). In other words, inthese embodiments, the device does not count the first person as being“present” unless the first person is oriented towards the display deviceand/or the first person is looking at the display device. If thesecriteria are not met, the device optionally determines that the firstperson is not paying attention to the playing of the media content inthe user interface, and modifies the playing of the media content, asdescribed above and/or below.

In some embodiments, the device reacts accordingly when two or morepeople are present at the display device. For example, while displayingthe media content to a single person (e.g., after detecting that thepresence criteria are met for the first person), the device optionallydetects that two or more people are present at the display device (e.g.,detecting that the presence criteria are met for a first person and asecond person). In accordance with a determination that two or morepeople are present at the display device, the device optionally modifiesthe playing of the media content in a second manner by increasing avisibility (e.g., size, brightness, and placement) of the media contentin the user interface. For example, the device optionally enlarges themedia content in the user interface to facilitate easier viewing by twoor more people concurrently, as each of the two or more people will, insome circumstances, need to be further from the display device than ifthere were only a single person watching.

In some circumstances, some or all of the media content is designated asprivate media content, and thus the device is configured to respondaccordingly when two or more people are present at the display device.For example, while displaying the media content to a single person(e.g., after detecting that the presence criteria are met for the firstperson), the device optionally detects that two or more people arepresent at the display device. In accordance with a determination thattwo or more people are present at the display device, the deviceoptionally determines whether the media content is designated as privatemedia content (e.g., not to be shared with more than the personassociated with it). In accordance with a determination that the mediacontent is designated as private media content, the device optionallymodifies the playing of the media content in a second manner by reducinga visibility of the media content (e.g., hiding, fading out, turning offthe display device) in the user interface. Thus, the device optionallyhinders the viewing of private media content by others.

In some embodiments, in accordance with a determination that the mediacontent is not designated as private media content (e.g., the mediacontent can be shared with more than the person associated with it), thedevice optionally modifies the playing of the media content in a thirdmanner by increasing the visibility (e.g., size, brightness, placement)of the media content in the user interface (e.g., enlarging the mediacontent in the user interface to facilitate easier viewing by two ormore people concurrently, as each of the two or more people will, insome circumstances, need to be further from the display device than ifthere were only a single person watching).

In some embodiments, when the one or more presence criteria aresatisfied, an action is optionally performed on the device based onactivity being performed on a second device. For example, in accordancewith a determination that the one or more presence criteria aresatisfied (e.g., the first person is present at the display device), thedevice optionally determines whether a second device (e.g., mobiledevice) is present at the display device with the first person, andwhether a specified type of activity (e.g., activities better suited forthe display device rather than the mobile device) is being performed onthe second device (e.g., mobile device). If so, in accordance with adetermination that the second device is present at the display devicewith the first person and that the specified type of activity is beingperformed on the second device, the device (the “first device”)optionally performs an action on the first device (e.g., a set top boxor other user interface generating device that is in communication witha remote control and a display device) based on the activity beingperformed on the second device.

For example, the first device optionally transfers activity (e.g.,playback of media content, video chatting) from the mobile device to thefirst device (e.g., set top box), and/or provides mobile device statusinformation (e.g., incoming phone call, new email notification, etc.) onthe display device (e.g., television). For example, if the first personis engaged in a video chat on a mobile device, when the first person andthe mobile device are present at the display device, the first deviceoptionally transfers the video chat from the mobile device to the firstdevice so that the first person can continue the video chat on thedisplay device, which could be a device such as a television. Such atransfer is also optionally performed for playback of media content(e.g., a movie) on the mobile device; specifically, transferringplayback of the media content from the mobile device to the first devicefor display on the display device. As another example, statusinformation about events on a mobile device are optionally provided onthe display device when the first person and the mobile device aredetermined to be present at the display device such that the firstperson can easily see this information without having to look at themobile device—status information such as information about incomingphone calls (e.g., caller name and number), new email messages received(e.g., sender name, subject line, first few lines of the email), newtext messages received (e.g., sender name, first few lines of the textmessage), etc.

In some embodiments, the device reverts playback of the media content toits original playback mode when the person returns to the displaydevice. For example, in some embodiments, at a first point in time, theone or more presence criteria are not satisfied (e.g., the person is notpresent at the display device), and the playing of the media content ismodified in the first manner (e.g., pausing the media content,bookmarking a current position in the media content, reducing a volumeof the media content). At a second point in time, after the modificationof the playing of the media content in the first manner, in someembodiments, the device determines that the one or more presencecriteria are satisfied (e.g., the person returns to the display device).In response to determining that the one or more presence criteria aresatisfied, the device optionally reverses the modification of theplaying of the media content in the first manner (e.g., resumingplayback of the media content, increasing the volume of the mediacontent, etc.).

FIGS. 15A-15C are flow diagrams illustrating a method 1500 of modifyingmedia content playback based on presence detection in accordance withsome embodiments (e.g., as described above with reference to FIGS.14A-14B). The method 1500 is performed at an electronic device such asdevice 500 as described above with reference to FIG. 5 , including a settop box or other user interface generating device that is incommunication with a remote control and a display device. Someoperations in method 1500 are, optionally, combined and/or the order ofsome operations is, optionally, changed.

As described below, the method 1500 provides ways in which a deviceoptionally modifies the playback of media content in a user interfacebased on presence detection. The method reduces the cognitive burden ona user when interacting with a user interface on the device, and reducesthe need for the user to physically interact with the device, therebycreating a more efficient human-machine interface. For battery-operatedelectronic devices, increasing the efficiency of the user's interactionwith the user interface conserves power and increases the time betweenbattery charges.

In some embodiments, an electronic device (e.g., a set top box or otheruser interface generating device that is in communication with a remotecontrol and a display device, such as device 1001 in FIG. 14A) with oneor more processors and memory (the “first device”) plays (1502) mediacontent (e.g., movie or music, such as the video presented in userinterface 1400 in FIG. 14A) on the first device. In some embodiments,the media content is included in a user interface (such as userinterface 1400 in FIG. 14A) for display on a display device (1504) (suchas display 1002 in FIG. 14A). In some embodiments, the first devicedetermines (1506) whether one or more presence criteria are satisfied.In some embodiments, the one or more presence criteria include acriterion that is satisfied while a first person is present at thedisplay device (1508). In some embodiments, the one or more presencecriteria include a criterion that is satisfied when the first person isoriented towards the display device (1510) (e.g., the first person'sface, shoulders, and/or torso are positioned towards the displaydevice). In some embodiments, the one or more presence criteria includea criterion that is satisfied when the first person's gaze is directedat the display device (1512) (e.g., the first person is looking at thedisplay device). In some embodiments, determining whether the one ormore presence criteria are satisfied is based on information receivedfrom one or more optical sensors (1514) (e.g., IR, depth, and/or RGBcameras).

In accordance with a determination that the one or more presencecriteria are not satisfied, in some embodiments, the first devicemodifies (1516) the playing of the media content in a first manner(e.g., pausing the media content, bookmarking a current position in themedia content, reducing a volume of the media content, such as isillustrated in FIG. 14B). In some embodiments, the first devicedetermines (1518) a second device (e.g., mobile device) associated withthe first person (e.g., the first person's iPhone). In some embodiments,modifying the playing of the media content in the first manner comprisestransferring the playing of the media content to the second device(1520) (e.g., transferring the playing of the media content from thedisplay device to the first person's iPhone in response to determiningthat the first person is no longer present at the display device, forexample by transferring an indication of progress of the user in thecontent so that the content will start playing on the second devicewhere the user left off on the first device). In some embodiments,transferring the playing of the media content includes initiatingplayback of the content on the second device. In some embodiments,transferring the playing of the media content includes transferring themedia content from the first device to the second device. In someembodiments, the playing of the media content is transferred to thesecond device without transferring the media content to the seconddevice (e.g., because the content is already present on both devices orbecause both devices are streaming the content from a remote source suchas a video or audio streaming service).

In some embodiments, in accordance with a determination that the one ormore presence criteria are satisfied (e.g., the first person is presentat the display device), the first device maintains (1524) the playing ofthe media content in the user interface (e.g., the first devicecontinues playing the media content when the first person is present atthe display device). In some embodiments, the first device determines(1526) a relative location of the first person with respect to thedisplay device (e.g., where in relation to the display device the firstperson is present). In some embodiments, the first device modifies(1528) the playing of the media content in a second manner (e.g.,beamforming audio associated with the media content) based on therelative location of the first person (e.g., the first device adjuststhe audio output of the media content to beamform the audio to the firstperson's position), wherein the second manner is different from thefirst manner. In some embodiments, the first device determines (1530) adistance of the first person from the display device. In someembodiments, the first device modifies (1532) the playing of the mediacontent in a second manner (e.g., beamforming audio, adjusting volume,adjusting the size of media content displayed) based on the distance ofthe first person from the display device, wherein the second manner isdifferent from the first manner. In some embodiments, the first devicedetermines (1534) whether a second device (e.g., mobile device) ispresent at the display device with the first person, and whether aspecified type of activity (e.g., activities better suited for thedisplay device rather than the mobile device) is being performed on thesecond device. In some embodiments, in accordance with a determinationthat the second device is present at the display device with the firstperson and that the specified type of activity is being performed on thesecond device, the first device (e.g., a set top box or other userinterface generating device that is in communication with a remotecontrol and a display device) performs (1536) an action on the firstdevice based on the activity being performed on the second device (e.g.,the first device transfers an activity (e.g., playback of media content,video chatting) from the mobile device to the first device (e.g., settop box), provides mobile device status information (e.g., incomingphone calls, new email notifications, etc.) on the display device (e.g.,television), etc.). For example, if the first person is engaged in avideo chat on a mobile device, when the first person and the mobiledevice are present at the display device, the first device transfers thevideo chat from the mobile device to the first device so that the firstperson can continue the video chat on the display device, which could bea device such as a television. Such a transfer is also optionallyperformed for playback of media content (e.g., a movie) on the mobiledevice; specifically, transferring playback of the media content fromthe mobile device to the first device for display on the display device.As another example, status information about events on a mobile deviceis optionally provided on the display device when the first person andthe mobile device are determined to be present at the display devicesuch that the first person can easily see this information withouthaving to look at the mobile device—status information is optionallyinformation such as information about incoming phone calls (e.g., callername and number), new email messages received (e.g., sender name,subject line, first few lines of the email), new text messages received(e.g., sender name, first few lines of the text message), etc.

In some embodiments, while displaying the media content to a singleperson (1538) (e.g., after detecting that the presence criteria are metfor the first person), the first device detects (1540) that two or morepeople are present at the display device (e.g., detecting that thepresence criteria are met for a first person and a second person). Insome embodiments, in accordance with a determination that two or morepeople are present at the display device (1542), the first devicemodifies (1544) the playing of the media content in a second manner byincreasing a visibility (e.g., size, brightness, placement) of the mediacontent in the user interface (e.g., enlarging the media content in theuser interface to facilitate easier viewing by two or more peopleconcurrently, as each of the two or more people will optionally need tobe further from the display device than if there were only a singleperson watching). In some embodiments, the first device determines(1546) whether the media content is designated as private media content(e.g., not to be shared with more than the person associated with it).In some embodiments, in accordance with a determination that the mediacontent is designated as private media content, the first devicemodifies (1548) the playing of the media content in a second manner byreducing a visibility of the media content (e.g., hiding, fading out,turning off the display device) in the user interface. In someembodiments, in accordance with a determination that the media contentis not designated as private media content (e.g., the media content canbe shared with more than the person associated with it), the firstdevice modifies (1550) the playing of the media content in a thirdmanner by increasing the visibility (e.g., size, brightness, placement)of the media content in the user interface (e.g., enlarging the mediacontent in the user interface to facilitate easier viewing by two ormore people concurrently, as each of the two or more people willoptionally need to be further from the display device than if there wereonly a single person watching).

In some embodiments, at a first point in time, the one or more presencecriteria are not satisfied (e.g., the person is not present at thedisplay device), and the playing of the media content is modified in thefirst manner (1552) (e.g., pausing the media content, bookmarking acurrent position in the media content, reducing a volume of the mediacontent). In some embodiments, at a second point in time, after themodification of the playing of the media content in the first manner,the first device determines (1554) that the one or more presencecriteria are satisfied (e.g., the person returns to the display device).In some embodiments, in response to determining that the one or morepresence criteria are satisfied, the first device reverses (1556) themodification of the playing of the media content in the first manner(e.g., resuming playback of the media content, increasing the volume ofthe media content, etc.).

It should be understood that the particular order in which theoperations in FIGS. 15A-15C have been described is merely exemplary andis not intended to indicate that the described order is the only orderin which the operations could be performed. One of ordinary skill in theart would recognize various ways to reorder the operations describedherein. Additionally, it should be noted that details of other processesdescribed herein with respect to other methods described herein (e.g.,methods 700, 900, 1100 and 1300) are also applicable in an analogousmanner to method 1500 described above with respect to FIGS. 15A-15C. Forexample, the user interfaces, display device, media content and opticalsensors described above with reference to method 1500 optionally haveone or more of the characteristics of the user interfaces, displaydevices, media content (or content items) and optical sensors describedherein with reference to other methods described herein (e.g., methods700, 900, 1100 and 1300). For brevity, these details are not repeatedhere.

The operations in the information processing methods described aboveare, optionally implemented by running one or more functional modules inan information processing apparatus such as general purpose processors(e.g., as described above with respect to FIGS. 1A, 3 and 5 ) orapplication specific chips.

The operations described above with reference to FIGS. 15A-15C are,optionally, implemented by components depicted in FIGS. 1A-1B. Forexample, playing operation 1502, determination operation 1506, andmodification operation 1516 are, optionally implemented by event sorter170, event recognizer 180, and event handler 190. Event monitor 171 inevent sorter 170 detects a contact on touch-sensitive display 112, andevent dispatcher module 174 delivers the event information toapplication 136-1. A respective event recognizer 180 of application136-1 compares the event information to respective event definitions186, and determines whether a first contact at a first location on thetouch-sensitive surface corresponds to a predefined event or sub-event,such as selection of an object on a user interface. When a respectivepredefined event or sub-event is detected, event recognizer 180activates an event handler 190 associated with the detection of theevent or sub-event. Event handler 190 optionally utilizes or calls dataupdater 176 or object updater 177 to update the application internalstate 192. In some embodiments, event handler 190 accesses a respectiveGUI updater 178 to update what is displayed by the application.Similarly, it would be clear to a person having ordinary skill in theart how other processes can be implemented based on the componentsdepicted in FIGS. 1A-1B.

In accordance with some embodiments, FIG. 16 shows a functional blockdiagram of an electronic device 1600 configured in accordance with theprinciples of the various described embodiments. The functional blocksof the device are, optionally, implemented by hardware, software, or acombination of hardware and software to carry out the principles of thevarious described embodiments. It is understood by persons of skill inthe art that the functional blocks described in FIG. 16 are, optionally,combined or separated into sub-blocks to implement the principles of thevarious described embodiments. Therefore, the description hereinoptionally supports any possible combination or separation or furtherdefinition of the functional blocks described herein.

As shown in FIG. 16 , an electronic device 1600 optionally includes areceiving unit 1602 configured to receive images, sensor units 1614configured to sense an orientation of the electronic device, and aprocessing unit 1604 coupled to the receiving unit 1602 and the sensorunits 1614. In some embodiments, the processing unit 1604 includes adisplay enabling unit 1606, an identifying unit 1608, a performing unit1610 and a determining unit 1612.

In some embodiments, the receiving unit 1602 is configured to receive afirst image including a face of a first person. In some embodiments, theprocessing unit 1604 is configured to generate a user interface fordisplay on a display device (e.g., with the display enabling unit 1606),identify the face of the first person from the first image (e.g., withthe identifying unit 1608), and perform an action on the electronicdevice in accordance with the identification of the face of the firstperson (e.g., with the performing unit 1610).

In some embodiments, the user interface includes a second image thatincludes a face of a second person, the face of the second person beingdifferent from the face of the first person, and the processing unit1604 is further configured to identify the face of the second personfrom the second image (e.g., with the identifying unit 1608), whereinperforming the action on the electronic device in accordance with theidentification of the face of the first person comprises replacing theface of the second person in the second image with the face of the firstperson in the first image (e.g., with the performing unit 1610). In someembodiments, the processing unit 1604 is further configured to determinean orientation of the face of the first person in the first image withrespect to the display device (e.g., with the determining unit 1612),wherein performing the action on the electronic device in accordancewith the identification of the face of the first person comprisesaligning the user interface for display on the display device with theorientation of the face of the first person in the first image (e.g.,with the performing unit 1610). In some embodiments, aligning the userinterface with the orientation of the face of the first person comprisesforgoing aligning the user interface based on an orientation of thedevice detected by one or more sensor units 1614 of the device (e.g.,with the performing unit 1610), the sensor units 1614 being coupled tothe processing unit 1604 and the receiving unit 1602.

In some embodiments, the user interface includes a content item beingplayed, and performing the action on the electronic device in accordancewith the identification of the face of the first person comprisesassociating the playing of the content item with the first person (e.g.,with the performing unit 1610). In some embodiments, the processing unit1604 is further configured to identify, based on the face of the firstperson in the first image, a first profile on the electronic device thatis associated with the face of the first person (e.g., with theidentifying unit 1608), wherein associating the playing of the contentitem with the first person comprises associating the playing of thecontent item with the first profile (e.g., with the performing unit1610). In some embodiments, associating the playing of the content itemwith the first profile comprises updating a play count of the contentitem in the first profile (e.g., with the performing unit 1610).

In some embodiments, the first image further includes a face of a secondperson, and the processing unit 1604 is further configured to identifythe face of the second person from the first image (e.g., with theidentifying unit 1608), identify, based on the face of the second personin the first image, a second profile on the electronic device that isassociated with the face of the second person (e.g., with theidentifying unit 1608), and associate the playing of the content itemwith the second profile (e.g., with the performing unit 1610).

In some embodiments, the receiving unit 1602 is further configured toreceive a second image including the face of the first person, andreceive a third image including the face of the first person and a faceof a second person, wherein the third image is captured after the secondimage was captured, and the processing unit 1604 is further configuredto identify the face of the first person from the second image (e.g.,with the identifying unit 1608), in response to identifying the face ofthe first person from the second image, continue to associate theplaying of the content item with the first person (e.g., with theperforming unit 1610), identify the face of the first person and theface of the second person from the third image (e.g., with theidentifying unit 1608), and in response to identifying the face of thefirst person and the face of the second person from the third image:continue to associate the playing of the content item with the firstperson (e.g., with the performing unit 1610), and associate the playingof the content item with the second person (e.g., with the performingunit 1610).

In some embodiments, the user interface includes a content item beingplayed, performing the action on the electronic device in accordancewith the identification of the face of the first person comprises makinga preliminarily association of the playing of the content item with thefirst person (e.g., with the performing unit 1610), the receiving unit1602 is further configured to receive a second image, wherein the secondimage is received at least a first time threshold after receiving thefirst image, and the processing unit 1604 is further configured to afterreceiving the second image, determine whether the second image includesthe face of the first person (e.g., with the determining unit 1612): inaccordance with a determination that the second image does not includethe face of the first person, cancel the preliminary association of theplaying of the content item with the first person (e.g., with theperforming unit 1610), and

in accordance with a determination that the second image includes theface of the first person, confirm the preliminary association of theplaying of the content item with the first person (e.g., with theperforming unit 1610).

In accordance with some embodiments, FIG. 17 shows a functional blockdiagram of an electronic device 1700 configured in accordance with theprinciples of the various described embodiments. The functional blocksof the device are, optionally, implemented by hardware, software, or acombination of hardware and software to carry out the principles of thevarious described embodiments. It is understood by persons of skill inthe art that the functional blocks described in FIG. 17 are, optionally,combined or separated into sub-blocks to implement the principles of thevarious described embodiments. Therefore, the description hereinoptionally supports any possible combination or separation or furtherdefinition of the functional blocks described herein.

As shown in FIG. 17 , an electronic device 1700 optionally includes areceiving unit 1702 configured to receive inputs, and a processing unit1704 coupled to the receiving unit 1702. In some embodiments, theprocessing unit 1704 includes a display enabling unit 1706, adetermining unit 1708, a performing unit 1710 and an interacting unit1712.

In some embodiments, the processing unit 1704 is configured to generatea user interface for display on a display device (e.g., with the displayenabling unit 1706), wherein the user interface includes a first userinterface element, determine a gaze location in the user interface(e.g., with the determining unit 1708), wherein the gaze locationcomprises a location in the user interface to which a person's gaze isdirected, determine whether the gaze location in the user interfacecoincides with the first user interface element (e.g., with thedetermining unit 1708), and in accordance with a determination that thegaze location coincides with the first user interface element, performan action with respect to the first user interface element (e.g., withthe performing unit 1710).

In some embodiments, the first user interface element comprises a userinterface element requesting attention, and performing the action withrespect to the first user interface element comprises acknowledging therequest for attention in response to determining that the gaze locationcoincides with the first user interface element (e.g., with theperforming unit 1710). In some embodiments, the user interface furtherincludes a second user interface element, and performing the action withrespect to the first user interface element comprises emphasizing thefirst user interface element relative to the second user interfaceelement (e.g., with the performing unit 1710).

In some embodiments, the processing unit 1704 is further configured todetermine one or more portions of the display device that are notcoincident with the gaze location and the first user interface element(e.g., with the determining unit 1708), wherein performing the actionwith respect to the first user interface element comprises adjusting anoutput of the one or more portions of the display device that are notcoincident with the gaze location (e.g., with the performing unit 1710).In some embodiments, performing the action with respect to the firstuser interface element comprises selecting the first user interfaceelement (e.g., with the performing unit 1710). In some embodiments, thefirst user interface element comprises a plurality of moving items, andthe processing unit 1704 is further configured to determine a first itemof the plurality of moving items with which the gaze location coincides(e.g., with the determining unit 1708), wherein performing the actionwith respect to the first user interface element comprises modifying amovement of the first item of the plurality of moving items such thatthe first item is visible in the user interface (e.g., with theperforming unit 1710). In some embodiments, the first user interfaceelement comprises an autoscrolling list, and the plurality of movingitems comprise elements of the autoscrolling list. In some embodiments,the first user interface element comprises an image catalog, and theplurality of moving items comprise images in the image catalog. In someembodiments, the first user interface element comprises a content bar,and the plurality of moving items comprise representations of contentitems accessible from the content bar.

In some embodiments, the first user interface element comprises content,and performing the action with respect to the first user interfaceelement comprises designating the content in the first user interfaceelement as being read (e.g., with the performing unit 1710). In someembodiments, performing the action with respect to the first userinterface element comprises designating the first user interface elementas an active user interface element (e.g., with the performing unit1710), the receiving unit 1702 is configured to receive a first inputfrom an input unit for interacting with an active user interfaceelement, the receiving unit being coupled to the processing unit,wherein the processing unit 1704 is further configured to in response tothe first input, interact with the first user interface element inaccordance with the first input (e.g., with the interacting unit 1712).In some embodiments, the user interface further includes a second userinterface element, a location of a cursor associated with the inputdevice is coincident with the second user interface element, anddesignating the first user interface element as the active userinterface element comprises ignoring the location of the cursor indesignating the first user interface element as the active userinterface element (e.g., with the performing unit 1710).

In accordance with some embodiments, FIG. 18 shows a functional blockdiagram of an electronic device 1800 configured in accordance with theprinciples of the various described embodiments. The functional blocksof the device are, optionally, implemented by hardware, software, or acombination of hardware and software to carry out the principles of thevarious described embodiments. It is understood by persons of skill inthe art that the functional blocks described in FIG. 18 are, optionally,combined or separated into sub-blocks to implement the principles of thevarious described embodiments. Therefore, the description hereinoptionally supports any possible combination or separation or furtherdefinition of the functional blocks described herein.

As shown in FIG. 18 , an electronic device 1800 optionally includes areceiving unit 1802 configured to receive inputs, a transmitting unit1804 configured to transmit, and a processing unit 1806 coupled to thereceiving unit 1802 and the transmitting unit 1804. In some embodiments,the processing unit 1806 includes a sound enabling unit 1808, adetermining unit 1810, a designating unit 1812, a detecting unit 1812and a transitioning unit 1816.

In some embodiments, the receiving unit 1802 is configured to receive anoutput from a microphone on a sound-receiving device, and the processingunit 1806 is configured to generate a first sound for playback on asound-playback device (e.g., with the sound enabling unit 1808),determine whether the sound-receiving device is present at thesound-playback device based on a plurality of presence criteria (e.g.,with the determining unit 1810), including sound-detection criteria thatis satisfied when the output from the microphone on the sound-receivingdevice corresponds to the first sound for playback on the sound-playbackdevice, and verification criteria that is satisfied when a presence ofthe sound-receiving device at the sound-playback device is verifiedusing a quantity other than the first sound, and in accordance with adetermination that the plurality of presence criteria are satisfied,designate a user of the sound-receiving device as being present at thesound-playback device (e.g., with the designating unit 1812).

In some embodiments, designating the user of the sound-receiving deviceas being present at the sound-playback device comprises determining auser associated with the sound-receiving device (e.g., with thedetermining unit 1810), and designating the user as being present at thesound-playback device (e.g., with the designating unit 1814). In someembodiments, the transmitting unit 1804 is coupled to the processingunit 1806 and the receiving unit 1802, and is configured to, prior tothe processing unit 1806 generating the first sound for playback on thesound-playback device, transmit a wake command to the sound-receivingdevice for turning on the microphone on the sound-receiving device.

In some embodiments, the verification criteria include a criterion thatis satisfied when a face of the user of the sound-receiving device isrecognized at the sound-playback device. In some embodiments, theverification criteria include a criterion that is satisfied when thesound-receiving device has sensed motion during a most recent timeperiod. In some embodiments, the verification criteria include acriterion that is satisfied when the sound-receiving device resides on asame wireless network as the sound-playback device.

In some embodiments, the processing unit 1806 is further configured todetect motion of the sound-receiving device (e.g., with the detectingunit 1814), and in accordance with a determination that the motionexceeds a predefined threshold, re-evaluate the plurality of presencecriteria to determine whether the sound-receiving device remains at thesound-playback device (e.g., with the determining unit 1810).

In some embodiments, the sound-playback device is configured totransition from a first state to a second state in response to one ormore transition criteria being satisfied at the sound-playback device,and the processing unit 1806 is further configured to, in accordancewith the determination that the plurality of presence criteria aresatisfied, forgo transitioning from the first state of thesound-playback device to the second state of the sound-playback devicein response to the one or more transition criteria being satisfied atthe sound-playback device (e.g., with the transitioning unit 1816).

In accordance with some embodiments, FIG. 19 shows a functional blockdiagram of an electronic device 1900 configured in accordance with theprinciples of the various described embodiments. The functional blocksof the device are, optionally, implemented by hardware, software, or acombination of hardware and software to carry out the principles of thevarious described embodiments. It is understood by persons of skill inthe art that the functional blocks described in FIG. 19 are, optionally,combined or separated into sub-blocks to implement the principles of thevarious described embodiments. Therefore, the description hereinoptionally supports any possible combination or separation or furtherdefinition of the functional blocks described herein.

As shown in FIG. 19 , an electronic device 1900 optionally includes areceiving unit 1902 configured to receive inputs, and a processing unit1904 coupled to the receiving unit 1902. In some embodiments, theprocessing unit 1904 includes a determining unit 1906, a displayenabling unit 1908, an identifying unit 1910, a detecting unit 1912 anda logging out unit 1914.

In some embodiments, the processing unit 1904 is configured to determinewhether a person is further than a threshold distance from a displaydevice or closer than the threshold distance from the display device(e.g., with the determining unit 1906), in accordance with adetermination that the person is further than the threshold distancefrom the display device, provide a first user interface for display onthe display device (e.g., with the display enabling unit 1908), and inaccordance with a determination that the person is closer than thethreshold distance from the display device, provide a second userinterface for display on the display device (e.g., with the displayenabling unit 1908), wherein the second user interface is different thanthe first user interface.

In some embodiments, determining whether the person is further than thethreshold distance from the display device or closer than the thresholddistance from the display device (e.g., with the determining unit 1906)is based on information received from one or more optical sensors,receiving unit 1902 coupled to the processing unit 1904 and configuredto receive the information from the one or more optical sensors. In someembodiments, the processing unit 1904 is further configured to identifya face of the person based on the information received from the one ormore optical sensors (e.g., with the identifying unit 1910), whereindetermining whether the person is further than the threshold distancefrom the display device or closer than the threshold distance from thedisplay device is based on a size of the face of the person (e.g., withthe determining unit 1906).

In some embodiments, the person is determined to be further than thethreshold distance from the display device (e.g., with the determiningunit 1906), and the processing unit 1904 is further configured to, whilethe first user interface is displayed on the display device, detectmovement of the person to a distance closer than the threshold distancefrom the display device (e.g., with the detecting unit 1912), and inresponse to detecting the movement of the person to the distance closerthan the threshold distance, provide the second user interface fordisplay on the display device (e.g., with the display enabling unit1908).

In some embodiments, the person is determined to be closer than thethreshold distance from the display device (e.g., with the determiningunit 1906), and the processing unit 1904 is further configured to, whilethe second user interface is displayed on the display device, detectmovement of the person to a distance further than the threshold distancefrom the display device (e.g., with the detecting unit 1912), and inresponse to detecting the movement of the person to the distance furtherthan the threshold distance, provide the first user interface fordisplay on the display device (e.g., with the display enabling unit1908).

In some embodiments, the person is further than the threshold distancefrom the display device, the first user interface is provided fordisplay on the displace device (e.g., with the display enabling unit1908), the first user interface including a first user interface elementand a second user interface element, and the processing unit 1904 isfurther configured to, while the first user interface is displayed onthe display device: determine a gaze location in the first userinterface (e.g., with the determining unit 1906), wherein the gazelocation comprises a location in the first user interface to which theperson's gaze is directed; detect movement of the person to a distancecloser than the threshold distance from the display device (e.g., withthe detecting nit 1912); and in response to detecting the movement ofthe person, provide the second user interface for display on the displaydevice (e.g., with the display enabling unit 1908), the second userinterface including information based on the gaze location.

In some embodiments, the processing unit 1904is further configured todetermine whether the gaze location coincides with the first userinterface element or the second user interface element in the first userinterface (e.g., with the determining unit 1906), and in response todetecting the movement of the person to the distance closer than thethreshold distance from the display device: in accordance with adetermination that the gaze location coincides with the first userinterface element, provide the second user interface for display on thedisplay device (e.g., with the display enabling unit 1908), the seconduser interface including information associated with the first userinterface element, and in accordance with a determination that the gazelocation coincides with the second user interface element, provide thesecond user interface for display on the display device (e.g., with thedisplay enabling unit 1908), the second user interface includinginformation associated with the second user interface element.

In some embodiments, the person is closer than the threshold distancefrom the display device, the person is logged into one or more areas onthe electronic device 1900, and the processing unit 1904 is furtherconfigured to detect movement of the person to a distance further thanthe threshold distance from the display device (e.g., with the detectingunit 1912), and in response to detecting the movement of the person tothe distance further than the threshold distance from the displaydevice, log out of the one or more areas on the electronic device (e.g.,logging out unit 1914).

In some embodiments, the first user interface includes a user interfaceelement at a first size, and the second user interface includes the userinterface element at a second size, smaller than the first size. In someembodiments, the first user interface includes summary information abouta first quantity, and the second user interface includes detailedinformation about the first quantity. In some embodiments, the firstuser interface includes a first set of user interface elements, and thesecond user interface includes the first set of user interface elementsand a second set of user interface elements.

In accordance with some embodiments, FIG. 20 shows a functional blockdiagram of a first device 2000 configured in accordance with theprinciples of the various described embodiments. The functional blocksof the device are, optionally, implemented by hardware, software, or acombination of hardware and software to carry out the principles of thevarious described embodiments. It is understood by persons of skill inthe art that the functional blocks described in FIG. 20 are, optionally,combined or separated into sub-blocks to implement the principles of thevarious described embodiments. Therefore, the description hereinoptionally supports any possible combination or separation or furtherdefinition of the functional blocks described herein.

As shown in FIG. 20 , a first device 2000 optionally includes atransferring unit 2002 configured to transfer operations to anotherdevice, a receiving unit 2004 configured to receive inputs, and aprocessing unit 2006 coupled to the transferring unit 2002 and thereceiving unit 2004. In some embodiments, the processing unit 2006includes a playing unit 2008, a determining unit 2010, a detecting unit2012 and a performing unit 2014.

In some embodiments, the processing unit 2006 is configured to playmedia content on the first device (e.g., with the playing unit 2006),wherein the media content is included in a user interface for display ona display device, determine whether one or more presence criteria aresatisfied (e.g., with the determining unit 2010), wherein the one ormore presence criteria include a criterion that is satisfied while afirst person is present at the display device, and in accordance with adetermination that the one or more presence criteria are not satisfied,modify the playing of the media content in a first manner (e.g., withthe playing unit 2008).

In some embodiments, the processing unit 2006 is further configured to,in accordance with a determination that the one or more presencecriteria are satisfied (e.g., with the determining unit 2010), maintainthe playing of the media content in the user interface (e.g., with theplaying unit 2008). In some embodiments, the processing unit 2006 isfurther configured to, in accordance with the determination that the oneor more presence criteria are not satisfied (e.g., with the determiningunit 2010), determine a second device associated with the first person(e.g., with the determining unit 2010), wherein modifying the playing ofthe media content in the first manner comprises transferring the playingof the media content to the second device, the transferring unit 2004configured to transfer the playing of the media content to the seconddevice.

In some embodiments, the processing unit 2006 is further configured to,in accordance with a determination that the one or more presencecriteria are satisfied (e.g., with the determining unit 2010): determinea relative location of the first person with respect to the displaydevice (e.g., with the determining unit 2010), and modify the playing ofthe media content in a second manner based on the relative location ofthe first person (e.g., with the playing unit 2008), wherein the secondmanner is different from the first manner. In some embodiments, theprocessing unit 2006 is further configured to, in accordance with adetermination that the one or more presence criteria are satisfied(e.g., with the determining unit 2010): determine a distance of thefirst person from the display device (e.g., with the determining unit2010), and modify the playing of the media content in a second mannerbased on the distance of the first person from the display device (e.g.,with the playing unit 2008), wherein the second manner is different fromthe first manner.

In some embodiments, the one or more presence criteria include acriterion that is satisfied when the first person is oriented towardsthe display device. In some embodiments, the one or more presencecriteria include a criterion that is satisfied when the first person'sgaze is directed at the display device.

In some embodiments, the processing unit 2006 is further configured to:while displaying the media content to a single person (e.g., with theplaying unit 2008), detect that two or more people are present at thedisplay device (e.g., with the detecting unit 2012), and in accordancewith a determination that two or more people are present at the displaydevice, modify the playing of the media content in a second manner byincreasing a visibility of the media content in the user interface(e.g., with the playing unit 2008).

In some embodiments, the processing unit 2006 is further configured to:while displaying the media content to a single person (e.g., with theplaying unit 2008), detect that two or more people are present at thedisplay device (e.g., with the detecting unit 2012), and in accordancewith a determination that two or more people are present at the displaydevice: determine whether the media content is designated as privatemedia content (e.g., with the determining unit 2010), and in accordancewith a determination that the media content is designated as privatemedia content, modify the playing of the media content in a secondmanner by reducing a visibility of the media content in the userinterface (e.g., with the playing unit 2008).

In some embodiments, the processing unit 2006 is further configured to,in accordance with a determination that the media content is notdesignated as private media content (e.g., with the determining unit2010), modify the playing of the media content in a third manner byincreasing the visibility of the media content in the user interface(e.g., with the playing unit 2008). In some embodiments, the processingunit 2006 is further configured to, in accordance with a determinationthat the one or more presence criteria are satisfied (e.g., with thedetermining unit 2010): determine whether a second device is present atthe display device with the first person, and whether a specified typeof activity is being performed on the second device (e.g., with thedetermining unit 2010), and in accordance with a determination that thesecond device is present at the display device with the first person andthat the specified type of activity is being performed on the seconddevice, perform an action on the first device based on the activitybeing performed on the second device (e.g., with the performing unit2014).

In some embodiments, determining whether the one or more presencecriteria are satisfied (e.g., with the determining unit 2010) is basedon information received from one or more optical sensors, the receivingunit 2004 configured to receive the information from the one or moreoptical sensors. In some embodiments, at a first point in time, the oneor more presence criteria are not satisfied, and the playing of themedia content is modified in the first manner (e.g., with the playingunit 2008), and the processing unit 2006 is further configured to: at asecond point in time, after the modification of the playing of the mediacontent in the first manner, determine that the one or more presencecriteria are satisfied (e.g., with the determining unit 2010), and inresponse to determining that the one or more presence criteria aresatisfied, reverse the modification of the playing of the media contentin the first manner (e.g., with the playing unit 2008).

The operations described above with reference to FIGS. 7A-7E, 9A-9C,11A-11B, 13A-13D and 15A-15C are, optionally, implemented by componentsdepicted in FIGS. 1A-1B, FIG. 16 , FIG. 17 , FIG. 18 , FIG. 19 , or FIG.20 . For example, determining operations 724, 920, 1110, 1302, and 1506are, optionally implemented by event sorter 170, event recognizer 180,and event handler 190. Event monitor 171 in event sorter 170 detects acontact on touch-sensitive display 112, and event dispatcher module 174delivers the event information to application 136-1. A respective eventrecognizer 180 of application 136-1 compares the event information torespective event definitions 186, and determines whether a first contactat a first location on the touch-sensitive surface corresponds to apredefined event or sub-event, such as selection of an object on a userinterface. When a respective predefined event or sub-event is detected,event recognizer 180 activates an event handler 190 associated with thedetection of the event or sub-event. Event handler 190 optionallyutilizes or calls data updater 176 or object updater 177 to update theapplication internal state 192. In some embodiments, event handler 190accesses a respective GUI updater 178 to update what is displayed by theapplication. Similarly, it would be clear to a person having ordinaryskill in the art how other processes can be implemented based on thecomponents depicted in FIGS. 1A-1B, FIGS. 1A-1B, FIG. 16 , FIG. 17 ,FIG. 18 , FIG. 19 , or FIG. 20 .

Therefore, according to the above, this disclosure includes subjectmatter directed to the following examples:

Example 1: A method comprising: at an electronic device with one or moreprocessors and memory: generating a user interface for display on adisplay device; receiving a first image including a face of a firstperson; identifying the face of the first person from the first image;and performing an action on the electronic device in accordance with theidentification of the face of the first person.

Example 2: The method of preceding example 1, wherein the user interfaceincludes a second image that includes a face of a second person, theface of the second person being different from the face of the firstperson, the method further comprising: identifying the face of thesecond person from the second image, wherein performing the action onthe electronic device in accordance with the identification of the faceof the first person comprises replacing the face of the second person inthe second image with the face of the first person in the first image.

Example 3: The method of any of examples 1-2, further comprising:determining an orientation of the face of the first person in the firstimage with respect to the display device, wherein performing the actionon the electronic device in accordance with the identification of theface of the first person comprises aligning the user interface fordisplay on the display device with the orientation of the face of thefirst person in the first image.

Example 4: The method of preceding example 3, wherein aligning the userinterface with the orientation of the face of the first person comprisesforgoing aligning the user interface based on an orientation of thedevice detected by one or more sensors of the device.

Example 5: The method of any of examples 1-4, wherein: the userinterface includes a content item being played, and performing theaction on the electronic device in accordance with the identification ofthe face of the first person comprises associating the playing of thecontent item with the first person.

Example 6: The method of preceding example 5, further comprising:identifying, based on the face of the first person in the first image, afirst profile on the electronic device that is associated with the faceof the first person, wherein associating the playing of the content itemwith the first person comprises associating the playing of the contentitem with the first profile.

Example 7: The method of preceding example 6, wherein associating theplaying of the content item with the first profile comprises updating aplay count of the content item in the first profile.

Example 8: The method of any of examples 5-7, wherein the first imagefurther includes a face of a second person, the method furthercomprising: identifying the face of the second person from the firstimage; identifying, based on the face of the second person in the firstimage, a second profile on the electronic device that is associated withthe face of the second person; and associating the playing of thecontent item with the second profile.

Example 9: The method of any of examples 5-8, further comprising:receiving a second image including the face of the first person;identifying the face of the first person from the second image; inresponse to identifying the face of the first person from the secondimage, continuing to associate the playing of the content item with thefirst person; receiving a third image including the face of the firstperson and a face of a second person, wherein the third image wascaptured after the second image was captured; identifying the face ofthe first person and the face of the second person from the third image;and in response to identifying the face of the first person and the faceof the second person from the third image: continuing to associate theplaying of the content item with the first person, and associating theplaying of the content item with the second person.

Example 10: The method of any of examples 1-9, wherein: the userinterface includes a content item being played, and performing theaction on the electronic device in accordance with the identification ofthe face of the first person comprises making a preliminarilyassociation of the playing of the content item with the first person;and the method further comprises: receiving a second image, wherein thesecond image is received at least a first time threshold after receivingthe first image; and after receiving the second image, determiningwhether the second image includes the face of the first person: inaccordance with a determination that the second image does not includethe face of the first person, canceling the preliminary association ofthe playing of the content item with the first person; and in accordancewith a determination that the second image includes the face of thefirst person, confirming the preliminary association of the playing ofthe content item with the first person.

Example 11: An electronic device, comprising: one or more processors;memory; and one or more programs, wherein the one or more programs arestored in the memory and configured to be executed by the one or moreprocessors, the one or more programs including instructions for:generating a user interface for display on a display device; receiving afirst image including a face of a first person; identifying the face ofthe first person from the first image; and performing an action on theelectronic device in accordance with the identification of the face ofthe first person.

Example 12: A non-transitory computer readable storage medium storingone or more programs, the one or more programs comprising instructions,which when executed by an electronic device with one or more processorsand memory, cause the device to: generate a user interface for displayon a display device; receive a first image including a face of a firstperson; identify the face of the first person from the first image; andperform an action on the electronic device in accordance with theidentification of the face of the first person.

Example 13: An electronic device, comprising: one or more processors;memory; means for generating a user interface for display on a displaydevice; means for receiving a first image including a face of a firstperson; means for identifying the face of the first person from thefirst image; and means for performing an action on the electronic devicein accordance with the identification of the face of the first person.

Example 14: An information processing apparatus for use in an electronicdevice with one or more processors and memory, comprising: means forgenerating a user interface for display on a display device; means forreceiving a first image including a face of a first person; means foridentifying the face of the first person from the first image; and meansfor performing an action on the electronic device in accordance with theidentification of the face of the first person.

Example 15: An electronic device, comprising: one or more processors;memory; and one or more programs, wherein the one or more programs arestored in the memory and configured to be executed by the one or moreprocessors, the one or more programs including instructions forperforming any of the methods of examples 1-10.

Example 16: A non-transitory computer readable storage medium storingone or more programs, the one or more programs comprising instructions,which when executed by an electronic device with one or more processorsand memory, cause the device to perform any of the methods of examples1-10.

Example 17: An electronic device, comprising: one or more processors;memory; and means for performing any of the methods of examples 1-10.

Example 18: An information processing apparatus for use in an electronicdevice with one or more processors and memory, comprising: means forperforming any of the method of examples 1-10.

Example 19: A method comprising: at an electronic device with one ormore processors and memory: generating a user interface for display on adisplay device, wherein the user interface includes a first userinterface element; determining a gaze location in the user interface,wherein the gaze location comprises a location in the user interface towhich a person's gaze is directed; determining whether the gaze locationin the user interface coincides with the first user interface element;and in accordance with a determination that the gaze location coincideswith the first user interface element, performing an action with respectto the first user interface element.

Example 20: The method of preceding example 19, wherein: the first userinterface element comprises a user interface element requestingattention, and performing the action with respect to the first userinterface element comprises acknowledging the request for attention inresponse to determining that the gaze location coincides with the firstuser interface element.

Example 21: The method of any of examples 19-20, wherein: the userinterface further includes a second user interface element, andperforming the action with respect to the first user interface elementcomprises emphasizing the first user interface element relative to thesecond user interface element.

Example 22: The method of any of examples 19-21, further comprising:determining one or more portions of the display device that are notcoincident with the gaze location and the first user interface element,wherein performing the action with respect to the first user interfaceelement comprises adjusting an output of the one or more portions of thedisplay device that are not coincident with the gaze location.

Example 23: The method of any of examples 19-22, wherein performing theaction with respect to the first user interface element comprisesselecting the first user interface element.

Example 24: The method of any of examples 19-23, wherein the first userinterface element comprises a plurality of moving items, the methodfurther comprising: determining a first item of the plurality of movingitems with which the gaze location coincides, wherein performing theaction with respect to the first user interface element comprisesmodifying a movement of the first item of the plurality of moving itemssuch that the first item is visible in the user interface.

Example 25: The method of preceding example 24, wherein the first userinterface element comprises an autoscrolling list, and the plurality ofmoving items comprise elements of the autoscrolling list.

Example 26: The method of any of examples 24-25, wherein the first userinterface element comprises an image catalog, and the plurality ofmoving items comprise images in the image catalog.

Example 27: The method of any of examples 24-26, wherein the first userinterface element comprises a content bar, and the plurality of movingitems comprise representations of content items accessible from thecontent bar.

Example 28: The method of any of examples 19-27, wherein: the first userinterface element comprises content, and performing the action withrespect to the first user interface element comprises designating thecontent in the first user interface element as being read.

Example 29: The method of any of examples 19-28, wherein performing theaction with respect to the first user interface element comprisesdesignating the first user interface element as an active user interfaceelement, the method further comprising: receiving a first input from aninput device for interacting with an active user interface element; andin response to the first input, interacting with the first userinterface element in accordance with the first input.

Example 30: The method of preceding example 29, wherein: the userinterface further includes a second user interface element, a locationof a cursor associated with the input device is coincident with thesecond user interface element, and designating the first user interfaceelement as the active user interface element comprises ignoring thelocation of the cursor in designating the first user interface elementas the active user interface element.

Example 31: An electronic device, comprising: one or more processors;memory; and one or more programs, wherein the one or more programs arestored in the memory and configured to be executed by the one or moreprocessors, the one or more programs including instructions for:generating a user interface for display on a display device, wherein theuser interface includes a first user interface element; determining agaze location in the user interface, wherein the gaze location comprisesa location in the user interface to which a person's gaze is directed;determining whether the gaze location in the user interface coincideswith the first user interface element; and in accordance with adetermination that the gaze location coincides with the first userinterface element, performing an action with respect to the first userinterface element.

Example 32: A non-transitory computer readable storage medium storingone or more programs, the one or more programs comprising instructions,which when executed by an electronic device with one or more processorsand memory, cause the device to: generate a user interface for displayon a display device, wherein the user interface includes a first userinterface element; determine a gaze location in the user interface,wherein the gaze location comprises a location in the user interface towhich a person's gaze is directed; determine whether the gaze locationin the user interface coincides with the first user interface element;and in accordance with a determination that the gaze location coincideswith the first user interface element, perform an action with respect tothe first user interface element.

Example 33: An electronic device, comprising: one or more processors;memory; means for generating a user interface for display on a displaydevice, wherein the user interface includes a first user interfaceelement; means for determining a gaze location in the user interface,wherein the gaze location comprises a location in the user interface towhich a person's gaze is directed; means for determining whether thegaze location in the user interface coincides with the first userinterface element; and means for, in accordance with a determinationthat the gaze location coincides with the first user interface element,performing an action with respect to the first user interface element.

Example 34: An information processing apparatus for use in an electronicdevice with one or more processors and memory, comprising: means forgenerating a user interface for display on a display device, wherein theuser interface includes a first user interface element; means fordetermining a gaze location in the user interface, wherein the gazelocation comprises a location in the user interface to which a person'sgaze is directed; means for determining whether the gaze location in theuser interface coincides with the first user interface element; andmeans for, in accordance with a determination that the gaze locationcoincides with the first user interface element, performing an actionwith respect to the first user interface element.

Example 35: An electronic device, comprising: one or more processors;memory; and one or more programs, wherein the one or more programs arestored in the memory and configured to be executed by the one or moreprocessors, the one or more programs including instructions forperforming any of the methods of examples 19-30.

Example 36: A non-transitory computer readable storage medium storingone or more programs, the one or more programs comprising instructions,which when executed by an electronic device with one or more processorsand memory, cause the device to perform any of the methods of examples19-30.

Example 37: An electronic device, comprising: one or more processors;memory; and means for performing any of the methods of examples 19-30.

Example 38: An information processing apparatus for use in an electronicdevice with one or more processors and memory, comprising: means forperforming any of the method of examples 19-30.

Example 39: A method comprising: at an electronic device with one ormore processors and memory: generating a first sound for playback on asound-playback device; receiving an output from a microphone on asound-receiving device; determining whether the sound-receiving deviceis present at the sound-playback device based on a plurality of presencecriteria, including: sound-detection criteria that is satisfied when theoutput from the microphone on the sound-receiving device corresponds tothe first sound for playback on the sound-playback device; andverification criteria that is satisfied when a presence of thesound-receiving device at the sound-playback device is verified using aquantity other than the first sound; and in accordance with adetermination that the plurality of presence criteria are satisfied,designating a user of the sound-receiving device as being present at thesound-playback device.

Example 40: The method of preceding example 39, wherein designating theuser of the sound-receiving device as being present at thesound-playback device comprises determining a user associated with thesound-receiving device, and designating the user as being present at thesound-playback device.

Example 41: The method of any of examples 39-40, further comprisingprior to generating the first sound for playback on the sound-playbackdevice, transmitting a wake command to the sound-receiving device forturning on the microphone on the sound-receiving device.

Example 42: The method of any of examples 39-41, wherein theverification criteria include a criterion that is satisfied when a faceof the user of the sound-receiving device is recognized at thesound-playback device.

Example 43: The method of any of examples 39-42, wherein theverification criteria include a criterion that is satisfied when thesound-receiving device has sensed motion during a most recent timeperiod.

Example 44: The method of any of examples 39-43, wherein theverification criteria include a criterion that is satisfied when thesound-receiving device resides on a same wireless network as thesound-playback device.

Example 45: The method of any of examples 39-44, further comprising:detecting motion of the sound-receiving device; and in accordance with adetermination that the motion exceeds a predefined threshold,re-evaluating the plurality of presence criteria to determine whetherthe sound-receiving device remains at the sound-playback device.

Example 46: The method of any of examples 39-45, wherein: thesound-playback device is configured to transition from a first state toa second state in response to one or more transition criteria beingsatisfied at the sound-playback device, the method further comprising:in accordance with the determination that the plurality of presencecriteria are satisfied, forgoing transitioning from the first state ofthe sound-playback device to the second state of the sound-playbackdevice in response to the one or more transition criteria beingsatisfied at the sound-playback device.

Example 47: An electronic device, comprising: one or more processors;memory; and one or more programs, wherein the one or more programs arestored in the memory and configured to be executed by the one or moreprocessors, the one or more programs including instructions for:generating a first sound for playback on a sound-playback device;receiving an output from a microphone on a sound-receiving device;determining whether the sound-receiving device is present at thesound-playback device based on a plurality of presence criteria,including: sound-detection criteria that is satisfied when the outputfrom the microphone on the sound-receiving device corresponds to thefirst sound for playback on the sound-playback device; and verificationcriteria that is satisfied when a presence of the sound-receiving deviceat the sound-playback device is verified using a quantity other than thefirst sound; and in accordance with a determination that the pluralityof presence criteria are satisfied, designating a user of thesound-receiving device as being present at the sound-playback device.

Example 48: A non-transitory computer readable storage medium storingone or more programs, the one or more programs comprising instructions,which when executed by an electronic device with one or more processorsand memory, cause the device to: generate a first sound for playback ona sound-playback device; receive an output from a microphone on asound-receiving device; determine whether the sound-receiving device ispresent at the sound-playback device based on a plurality of presencecriteria, including: sound-detection criteria that is satisfied when theoutput from the microphone on the sound-receiving device corresponds tothe first sound for playback on the sound-playback device; andverification criteria that is satisfied when a resence of thesound-receiving device at the sound-playback device is verified using aquantity other than the first sound; and in accordance with adetermination that the plurality of presence criteria are satisfied,designate a user of the sound-receiving device as being present at thesound-playback device.

Example 49: An electronic device, comprising: one or more processors;memory; means for generating a first sound for playback on asound-playback device; means for receiving an output from a microphoneon a sound-receiving device; means for determining whether thesound-receiving device is present at the sound-playback device based ona plurality of presence criteria, including: sound-detection criteriathat is satisfied when the output from the microphone on thesound-receiving device corresponds to the first sound for playback onthe sound-playback device; and verification criteria that is satisfiedwhen a presence of the sound-receiving device at the sound-playbackdevice is verified using a quantity other than the first sound; andmeans for, in accordance with a determination that the plurality ofpresence criteria are satisfied, designating a user of thesound-receiving device as being present at the sound-playback device.

Example 50: An information processing apparatus for use in an electronicdevice with one or more processors and memory, comprising: means forgenerating a first sound for playback on a sound-playback device; meansfor receiving an output from a microphone on a sound-receiving device;means for determining whether the sound-receiving device is present atthe sound-playback device based on a plurality of presence criteria,including: sound-detection criteria that is satisfied when the outputfrom the microphone on the sound-receiving device corresponds to thefirst sound for playback on the sound-playback device; and verificationcriteria that is satisfied when a presence of the sound-receiving deviceat the sound-playback device is verified using a quantity other than thefirst sound; and means for, in accordance with a determination that theplurality of presence criteria are satisfied, designating a user of thesound-receiving device as being present at the sound-playback device.

Example 51: An electronic device, comprising: one or more processors;memory; and one or more programs, wherein the one or more programs arestored in the memory and configured to be executed by the one or moreprocessors, the one or more programs including instructions forperforming any of the methods of examples 39-46.

Example 52: A non-transitory computer readable storage medium storingone or more programs, the one or more programs comprising instructions,which when executed by an electronic device with one or more processorsand memory, cause the device to perform any of the methods of examples39-46.

Example 53: An electronic device, comprising: one or more processors;memory; and means for performing any of the methods of examples 39-46.

Example 54: An information processing apparatus for use in an electronicdevice with one or more processors and memory, comprising: means forperforming any of the method of examples 39-46.

Example 55: A method comprising: at an electronic device with one ormore processors and memory: determining whether a person is further thana threshold distance from a display device or closer than the thresholddistance from the display device; in accordance with a determinationthat the person is further than the threshold distance from the displaydevice, providing a first user interface for display on the displaydevice; and in accordance with a determination that the person is closerthan the threshold distance from the display device, providing a seconduser interface for display on the display device, wherein the seconduser interface is different than the first user interface.

Example 56: The method of preceding example 55, wherein determiningwhether the person is further than the threshold distance from thedisplay device or closer than the threshold distance from the displaydevice is based on information received from one or more opticalsensors.

Example 57: The method of preceding example 56, further comprisingidentifying a face of the person based on the information received fromthe one or more optical sensors, wherein determining whether the personis further than the threshold distance from the display device or closerthan the threshold distance from the display device is based on a sizeof the face of the person.

Example 58: The method of any of examples 55-57, wherein the person isdetermined to be further than the threshold distance from the displaydevice, the method further comprising: while the first user interface isdisplayed on the display device, detecting movement of the person to adistance closer than the threshold distance from the display device; andin response to detecting the movement of the person to the distancecloser than the threshold distance, providing the second user interfacefor display on the display device.

Example 59: The method of any of examples 55-57, wherein the person isdetermined to be closer than the threshold distance from the displaydevice, the method further comprising: while the second user interfaceis displayed on the display device, detecting movement of the person toa distance further than the threshold distance from the display device;and in response to detecting the movement of the person to the distancefurther than the threshold distance, providing the first user interfacefor display on the display device.

Example 60: The method of any of examples 55-58, wherein: the person isfurther than the threshold distance from the display device, and thefirst user interface is provided for display on the displace device, thefirst user interface including a first user interface element and asecond user interface element, the method further comprising: while thefirst user interface is displayed on the display device: determining agaze location in the first user interface, wherein the gaze locationcomprises a location in the first user interface to which the person'sgaze is directed; detecting movement of the person to a distance closerthan the threshold distance from the display device; and in response todetecting the movement of the person, providing the second userinterface for display on the display device, the second user interfaceincluding information based on the gaze location.

Example 61: The method of preceding example 60, further comprising:determining whether the gaze location coincides with the first userinterface element or the second user interface element in the first userinterface, wherein in response to detecting the movement of the personto the distance closer than the threshold distance from the displaydevice: in accordance with a determination that the gaze locationcoincides with the first user interface element, providing the seconduser interface for display on the display device, the second userinterface including information associated with the first user interfaceelement, and in accordance with a determination that the gaze locationcoincides with the second user interface element, providing the seconduser interface for display on the display device, the second userinterface including information associated with the second userinterface element.

Example 62: The method of any of examples 55-57 and 59, wherein: theperson is closer than the threshold distance from the display device,and the person is logged into one or more areas on the electronicdevice, the method further comprising: detecting movement of the personto a distance further than the threshold distance from the displaydevice; and in response to detecting the movement of the person to thedistance further than the threshold distance from the display device,logging out of the one or more areas on the electronic device.

Example 63: The method of any of examples 55-62, wherein the first userinterface includes a user interface element at a first size, and thesecond user interface includes the user interface element at a secondsize, smaller than the first size.

Example 64: The method of any of examples 55-63, wherein the first userinterface includes summary information about a first quantity, and thesecond user interface includes detailed information about the firstquantity.

Example 65: The method of any of examples 55-64, wherein the first userinterface includes a first set of user interface elements, and thesecond user interface includes the first set of user interface elementsand a second set of user interface elements.

Example 66: An electronic device, comprising: one or more processors;memory; and one or more programs, wherein the one or more programs arestored in the memory and configured to be executed by the one or moreprocessors, the one or more programs including instructions for:determining whether a person is further than a threshold distance from adisplay device or closer than the threshold distance from the displaydevice; in accordance with a determination that the person is furtherthan the threshold distance from the display device, providing a firstuser interface for display on the display device; and in accordance witha determination that the person is closer than the threshold distancefrom the display device, providing a second user interface for displayon the display device, wherein the second user interface is differentthan the first user interface.

Example 67: A non-transitory computer readable storage medium storingone or more programs, the one or more programs comprising instructions,which when executed by an electronic device with one or more processorsand memory, cause the device to: determine whether a person is furtherthan a threshold distance from a display device or closer than thethreshold distance from the display device; in accordance with adetermination that the person is further than the threshold distancefrom the display device, provide a first user interface for display onthe display device; and in accordance with a determination that theperson is closer than the threshold distance from the display device,provide a second user interface for display on the display device,wherein the second user interface is different than the first userinterface.

Example 68: An electronic device, comprising: one or more processors;memory; means for determining whether a person is further than athreshold distance from a display device or closer than the thresholddistance from the display device; means for, in accordance with adetermination that the person is further than the threshold distancefrom the display device, providing a first user interface for display onthe display device; and means for, in accordance with a determinationthat the person is closer than the threshold distance from the displaydevice, providing a second user interface for display on the displaydevice, wherein the second user interface is different than the firstuser interface.

Example 69: An information processing apparatus for use in an electronicdevice with one or more processors and memory, comprising: means fordetermining whether a person is further than a threshold distance from adisplay device or closer than the threshold distance from the displaydevice; means for, in accordance with a determination that the person isfurther than the threshold distance from the display device, providing afirst user interface for display on the display device; and means for,in accordance with a determination that the person is closer than thethreshold distance from the display device, providing a second userinterface for display on the display device, wherein the second userinterface is different than the first user interface.

Example 70: An electronic device, comprising: one or more processors;memory; and one or more programs, wherein the one or more programs arestored in the memory and configured to be executed by the one or moreprocessors, the one or more programs including instructions forperforming any of the methods of examples 55-65.

Example 71: A non-transitory computer readable storage medium storingone or more programs, the one or more programs comprising instructions,which when executed by an electronic device with one or more processorsand memory, cause the device to perform any of the methods of examples55-65.

Example 72: An electronic device, comprising: one or more processors;memory; and means for performing any of the methods of examples 55-65.

Example 73: An information processing apparatus for use in an electronicdevice with one or more processors and memory, comprising: means forperforming any of the method of examples 55-65.

Example 74: A method comprising: at a first device with one or moreprocessors and memory: playing media content on the first device,wherein the media content is included in a user interface for display ona display device; determining whether one or more presence criteria aresatisfied, wherein the one or more presence criteria include a criterionthat is satisfied while a first person is present at the display device;and in accordance with a determination that the one or more presencecriteria are not satisfied, modifying the playing of the media contentin a first manner.

Example 75: The method of preceding example 74, further comprising inaccordance with a determination that the one or more presence criteriaare satisfied, maintaining the playing of the media content in the userinterface.

Example 76: The method of any of examples 74-75, further comprising: inaccordance with the determination that the one or more presence criteriaare not satisfied, determining a second device associated with the firstperson, wherein modifying the playing of the media content in the firstmanner comprises transferring the playing of the media content to thesecond device.

Example 77: The method of any of examples 74-76, further comprising inaccordance with a determination that the one or more presence criteriaare satisfied: determining a relative location of the first person withrespect to the display device, and modifying the playing of the mediacontent in a second manner based on the relative location of the firstperson, wherein the second manner is different from the first manner.

Example 78: The method of any of examples 74-77, further comprising inaccordance with a determination that the one or more presence criteriaare satisfied: determining a distance of the first person from thedisplay device, and modifying the playing of the media content in asecond manner based on the distance of the first person from the displaydevice, wherein the second manner is different from the first manner.

Example 79: The method of any of examples 74-78, wherein the one or morepresence criteria include a criterion that is satisfied when the firstperson is oriented towards the display device.

Example 80: The method of any of examples 74-79, wherein the one or morepresence criteria include a criterion that is satisfied when the firstperson's gaze is directed at the display device.

Example 81: The method of any of examples 74-80, further comprising:while displaying the media content to a single person, detecting thattwo or more people are present at the display device; and in accordancewith a determination that two or more people are present at the displaydevice, modifying the playing of the media content in a second manner byincreasing a visibility of the media content in the user interface.

Example 82: The method of any of examples 74-80, further comprising:while displaying the media content to a single person, detecting thattwo or more people are present at the display device; and in accordancewith a determination that two or more people are present at the displaydevice: determining whether the media content is designated as privatemedia content, and in accordance with a determination that the mediacontent is designated as private media content, modifying the playing ofthe media content in a second manner by reducing a visibility of themedia content in the user interface.

Example 83: The method of preceding example 82, further comprising: inaccordance with a determination that the media content is not designatedas private media content, modifying the playing of the media content ina third manner by increasing the visibility of the media content in theuser interface.

Example 84: The method of any of examples 74-83, further comprising inaccordance with a determination that the one or more presence criteriaare satisfied: determining whether a second device is present at thedisplay device with the first person, and whether a specified type ofactivity is being performed on the second device; and in accordance witha determination that the second device is present at the display devicewith the first person and that the specified type of activity is beingperformed on the second device, performing an action on the first devicebased on the activity being performed on the second device.

Example 85: The method of any of examples 74-84, wherein determiningwhether the one or more presence criteria are satisfied is based oninformation received from one or more optical sensors.

Example 86: The method of any of examples 74-85, wherein: at a firstpoint in time, the one or more presence criteria are not satisfied, andthe playing of the media content is modified in the first manner, themethod further comprising: at a second point in time, after themodification of the playing of the media content in the first manner,determining that the one or more presence criteria are satisfied; and inresponse to determining that the one or more presence criteria aresatisfied, reversing the modification of the playing of the mediacontent in the first manner.

Example 87: A first electronic device, comprising: one or moreprocessors; memory; and one or more programs, wherein the one or moreprograms are stored in the memory and configured to be executed by theone or more processors, the one or more programs including instructionsfor: playing media content on the first device, wherein the mediacontent is included in a user interface for display on a display device;determining whether one or more presence criteria are satisfied, whereinthe one or more presence criteria include a criterion that is satisfiedwhile a first person is present at the display device; and in accordancewith a determination that the one or more presence criteria are notsatisfied, modifying the playing of the media content in a first manner.

Example 88: A non-transitory computer readable storage medium storingone or more programs, the one or more programs comprising instructions,which when executed by a first electronic device with one or moreprocessors and memory, cause the first device to: play media content onthe first device, wherein the media content is included in a userinterface for display on a display device; determine whether one or morepresence criteria are satisfied, wherein the one or more presencecriteria include a criterion that is satisfied while a first person ispresent at the display device; and in accordance with a determinationthat the one or more presence criteria are not satisfied, modify theplaying of the media content in a first manner.

Example 89: A first electronic device, comprising: one or moreprocessors; memory; means for playing media content on the first device,wherein the media content is included in a user interface for display ona display device; means for determining whether one or more presencecriteria are satisfied, wherein the one or more presence criteriainclude a criterion that is satisfied while a first person is present atthe display device; and means for, in accordance with a determinationthat the one or more presence criteria are not satisfied, modifying theplaying of the media content in a first manner.

Example 90: An information processing apparatus for use in a firstelectronic device with one or more processors and memory, comprising:means for playing media content on the first device, wherein the mediacontent is included in a user interface for display on a display device;means for determining whether one or more presence criteria aresatisfied, wherein the one or more presence criteria include a criterionthat is satisfied while a first person is present at the display device;and means for, in accordance with a determination that the one or morepresence criteria are not satisfied, modifying the playing of the mediacontent in a first manner.

Example 91: An electronic device, comprising: one or more processors;memory; and one or more programs, wherein the one or more programs arestored in the memory and configured to be executed by the one or moreprocessors, the one or more programs including instructions forperforming any of the methods of examples 74-86.

Example 92: A non-transitory computer readable storage medium storingone or more programs, the one or more programs comprising instructions,which when executed by an electronic device with one or more processorsand memory, cause the device to perform any of the methods of examples74-86.

Example 93: An electronic device, comprising: one or more processors;memory; and means for performing any of the methods of examples 74-86.

Example 94: An information processing apparatus for use in an electronicdevice with one or more processors and memory, comprising: means forperforming any of the method of examples 74-86.

Example 95: An electronic device, comprising: a receiving unitconfigured to receive a first image including a face of a first person;and a processing unit coupled to the receiving unit, the processing unitconfigured to: generate a user interface for display on a displaydevice; identify the face of the first person from the first image; andperform an action on the electronic device in accordance with theidentification of the face of the first person.

Example 96: The electronic device of preceding example 95, wherein theuser interface includes a second image that includes a face of a secondperson, the face of the second person being different from the face ofthe first person, and the processing unit is further configured to:identify the face of the second person from the second image, whereinperforming the action on the electronic device in accordance with theidentification of the face of the first person comprises replacing theface of the second person in the second image with the face of the firstperson in the first image.

Example 97: The electronic device of any of examples 95-96, wherein theprocessing unit is further configured to: determine an orientation ofthe face of the first person in the first image with respect to thedisplay device, wherein performing the action on the electronic devicein accordance with the identification of the face of the first personcomprises aligning the user interface for display on the display devicewith the orientation of the face of the first person in the first image.

Example 98: The electronic device of preceding example 97, whereinaligning the user interface with the orientation of the face of thefirst person comprises forgoing aligning the user interface based on anorientation of the device detected by one or more sensor units of thedevice, the sensor units being coupled to the processing unit and thereceiving unit.

Example 99: The electronic device of any of examples 95-98, wherein: theuser interface includes a content item being played, and performing theaction on the electronic device in accordance with the identification ofthe face of the first person comprises associating the playing of thecontent item with the first person.

Example 100: The electronic device of preceding example 99, wherein theprocessing unit is further configured to: identify, based on the face ofthe first person in the first image, a first profile on the electronicdevice that is associated with the face of the first person, whereinassociating the playing of the content item with the first personcomprises associating the playing of the content item with the firstprofile.

Example 101: The electronic device of preceding example 100, whereinassociating the playing of the content item with the first profilecomprises updating a play count of the content item in the firstprofile.

Example 102: The electronic device of any of examples 99-101, whereinthe first image further includes a face of a second person, and theprocessing unit is further configured to: identify the face of thesecond person from the first image; identify, based on the face of thesecond person in the first image, a second profile on the electronicdevice that is associated with the face of the second person; andassociate the playing of the content item with the second profile.

Example 103: The electronic device of any of examples 99-102, wherein:the receiving unit is further configured to receive a second imageincluding the face of the first person, and receive a third imageincluding the face of the first person and a face of a second person,wherein the third image is captured after the second image was captured,and the processing unit is further configured to: identify the face ofthe first person from the second image; in response to identifying theface of the first person from the second image, continue to associatethe playing of the content item with the first person; identify the faceof the first person and the face of the second person from the thirdimage; and in response to identifying the face of the first person andthe face of the second person from the third image: continue toassociate the playing of the content item with the first person, andassociate the playing of the content item with the second person.

Example 104: The electronic device of any of examples 95-103, wherein:the user interface includes a content item being played, performing theaction on the electronic device in accordance with the identification ofthe face of the first person comprises making a preliminarilyassociation of the playing of the content item with the first person,the receiving unit is further configured to receive a second image,wherein the second image is received at least a first time thresholdafter receiving the first image, and the processing unit is furtherconfigured to: after receiving the second image, determine whether thesecond image includes the face of the first person: in accordance with adetermination that the second image does not include the face of thefirst person, cancel the preliminary association of the playing of thecontent item with the first person; and in accordance with adetermination that the second image includes the face of the firstperson, confirm the preliminary association of the playing of thecontent item with the first person.

Example 105: An electronic device, comprising: a processing unitconfigured to: generate a user interface for display on a displaydevice, wherein the user interface includes a first user interfaceelement; determine a gaze location in the user interface, wherein thegaze location comprises a location in the user interface to which aperson's gaze is directed; determine whether the gaze location in theuser interface coincides with the first user interface element; and inaccordance with a determination that the gaze location coincides withthe first user interface element, perform an action with respect to thefirst user interface element.

Example 106: The electronic device of preceding example 105, wherein:the first user interface element comprises a user interface elementrequesting attention, and performing the action with respect to thefirst user interface element comprises acknowledging the request forattention in response to determining that the gaze location coincideswith the first user interface element.

Example 107: The electronic device of any of examples 105-106, wherein:the user interface further includes a second user interface element, andperforming the action with respect to the first user interface elementcomprises emphasizing the first user interface element relative to thesecond user interface element.

Example 108: The electronic device of any of examples 105-107, whereinthe processing unit is further configured to: determine one or moreportions of the display device that are not coincident with the gazelocation and the first user interface element, wherein performing theaction with respect to the first user interface element comprisesadjusting an output of the one or more portions of the display devicethat are not coincident with the gaze location.

Example 109: The electronic device of any of examples 105-108, whereinperforming the action with respect to the first user interface elementcomprises selecting the first user interface element.

Example 110: The electronic device of any of examples 105-109, whereinthe first user interface element comprises a plurality of moving items,and the processing unit is further configured to: determine a first itemof the plurality of moving items with which the gaze location coincides,wherein performing the action with respect to the first user interfaceelement comprises modifying a movement of the first item of theplurality of moving items such that the first item is visible in theuser interface.

Example 111: The electronic device of preceding example 110, wherein thefirst user interface element comprises an autoscrolling list, and theplurality of moving items comprise elements of the autoscrolling list.

Example 112: The electronic device of any of examples 110-111, whereinthe first user interface element comprises an image catalog, and theplurality of moving items comprise images in the image catalog.

Example 113: The electronic device of any of examples 110-112, whereinthe first user interface element comprises a content bar, and theplurality of moving items comprise representations of content itemsaccessible from the content bar.

Example 114: The electronic device of any of examples 105-113, wherein:the first user interface element comprises content, and performing theaction with respect to the first user interface element comprisesdesignating the content in the first user interface element as beingread.

Example 115: The electronic device of any of examples 105-114, whereinperforming the action with respect to the first user interface elementcomprises designating the first user interface element as an active userinterface element, the electronic device further comprising: a receivingunit configured to receive a first input from an input unit forinteracting with an active user interface element, the receiving unitbeing coupled to the processing unit, wherein the processing unit isfurther configured to: in response to the first input, interact with thefirst user interface element in accordance with the first input.

Example 116: The electronic device of preceding example 115, wherein:the user interface further includes a second user interface element, alocation of a cursor associated with the input device is coincident withthe second user interface element, and designating the first userinterface element as the active user interface element comprisesignoring the location of the cursor in designating the first userinterface element as the active user interface element.

Example 117: An electronic device, comprising: a receiving unitconfigured to receive an output from a microphone on a sound-receivingdevice; and a processing unit coupled to the receiving unit andconfigured to: generate a first sound for playback on a sound-playbackdevice; determine whether the sound-receiving device is present at thesound-playback device based on a plurality of presence criteria,including: sound-detection criteria that is satisfied when the outputfrom the microphone on the sound-receiving device corresponds to thefirst sound for playback on the sound-playback device; and verificationcriteria that is satisfied when a presence of the sound-receiving deviceat the sound-playback device is verified using a quantity other than thefirst sound; and in accordance with a determination that the pluralityof presence criteria are satisfied, designate a user of thesound-receiving device as being present at the sound-playback device.

Example 118: The electronic device of preceding example 117, whereindesignating the user of the sound-receiving device as being present atthe sound-playback device comprises determining a user associated withthe sound-receiving device, and designating the user as being present atthe sound-playback device.

Example 119: The electronic device of any of examples 117-118, furthercomprising a transmitting unit coupled to the processing unit and thereceiving unit, and configured to, prior to the processing unitgenerating the first sound for playback on the sound-playback device,transmit a wake command to the sound-receiving device for turning on themicrophone on the sound-receiving device.

Example 120: The electronic device of any of examples 117-119, whereinthe verification criteria include a criterion that is satisfied when aface of the user of the sound-receiving device is recognized at thesound-playback device.

Example 121: The electronic device of any of examples 117-120, whereinthe verification criteria include a criterion that is satisfied when thesound-receiving device has sensed motion during a most recent timeperiod.

Example 122: The electronic device of any of examples 117-121, whereinthe verification criteria include a criterion that is satisfied when thesound-receiving device resides on a same wireless network as thesound-playback device.

Example 123: The electronic device of any of examples 117-122, whereinthe processing unit is further configured to: detect motion of thesound-receiving device; and in accordance with a determination that themotion exceeds a predefined threshold, re-evaluate the plurality ofpresence criteria to determine whether the sound-receiving deviceremains at the sound-playback device.

Example 124: The electronic device of any of examples 117-123, wherein:the sound-playback device is configured to transition from a first stateto a second state in response to one or more transition criteria beingsatisfied at the sound-playback device, and the processing unit isfurther configured to: in accordance with the determination that theplurality of presence criteria are satisfied, forgo transitioning fromthe first state of the sound-playback device to the second state of thesound-playback device in response to the one or more transition criteriabeing satisfied at the sound-playback device.

Example 125: An electronic device, comprising: a processing unitconfigured to: determine whether a person is further than a thresholddistance from a display device or closer than the threshold distancefrom the display device; in accordance with a determination that theperson is further than the threshold distance from the display device,provide a first user interface for display on the display device; and inaccordance with a determination that the person is closer than thethreshold distance from the display device, provide a second userinterface for display on the display device, wherein the second userinterface is different than the first user interface.

Example 126: The electronic device of preceding example 125, whereindetermining whether the person is further than the threshold distancefrom the display device or closer than the threshold distance from thedisplay device is based on information received from one or more opticalsensors, the electronic device further comprising a receiving unitcoupled to the processing unit and configured to receive the informationfrom the one or more optical sensors.

Example 127: The electronic device of preceding example 126, wherein theprocessing unit is further configured to identify a face of the personbased on the information received from the one or more optical sensors,wherein determining whether the person is further than the thresholddistance from the display device or closer than the threshold distancefrom the display device is based on a size of the face of the person.

Example 128: The electronic device of any of examples 125-127, whereinthe person is determined to be further than the threshold distance fromthe display device, and the processing unit is further configured to:while the first user interface is displayed on the display device,detect movement of the person to a distance closer than the thresholddistance from the display device; and in response to detecting themovement of the person to the distance closer than the thresholddistance, provide the second user interface for display on the displaydevice.

Example 129: The electronic device of any of examples 125-127, whereinthe person is determined to be closer than the threshold distance fromthe display device, and the processing unit is further configured to:while the second user interface is displayed on the display device,detect movement of the person to a distance further than the thresholddistance from the display device; and in response to detecting themovement of the person to the distance further than the thresholddistance, provide the first user interface for display on the displaydevice.

Example 130: The electronic device of any of examples 125-128, wherein:the person is further than the threshold distance from the displaydevice, the first user interface is provided for display on the displacedevice, the first user interface including a first user interfaceelement and a second user interface element, and the processing unit isfurther configured to: while the first user interface is displayed onthe display device: determine a gaze location in the first userinterface, wherein the gaze location comprises a location in the firstuser interface to which the person's gaze is directed; detect movementof the person to a distance closer than the threshold distance from thedisplay device; and in response to detecting the movement of the person,provide the second user interface for display on the display device, thesecond user interface including information based on the gaze location.

Example 131: The electronic device of preceding example 130, wherein theprocessing unit is further configured to: determine whether the gazelocation coincides with the first user interface element or the seconduser interface element in the first user interface, and in response todetecting the movement of the person to the distance closer than thethreshold distance from the display device: in accordance with adetermination that the gaze location coincides with the first userinterface element, provide the second user interface for display on thedisplay device, the second user interface including informationassociated with the first user interface element, and in accordance witha determination that the gaze location coincides with the second userinterface element, provide the second user interface for display on thedisplay device, the second user interface including informationassociated with the second user interface element.

Example 132: The electronic device of any of examples 125-127 and 129,wherein: the person is closer than the threshold distance from thedisplay device, the person is logged into one or more areas on theelectronic device, and the processing unit is further configured to:detect movement of the person to a distance further than the thresholddistance from the display device; and in response to detecting themovement of the person to the distance further than the thresholddistance from the display device, log out of the one or more areas onthe electronic device.

Example 133: The electronic device of any of examples 125-132, whereinthe first user interface includes a user interface element at a firstsize, and the second user interface includes the user interface elementat a second size, smaller than the first size.

Example 134: The electronic device of any of examples 125-133, whereinthe first user interface includes summary information about a firstquantity, and the second user interface includes detailed informationabout the first quantity.

Example 135: The electronic device of any of examples 125-134, whereinthe first user interface includes a first set of user interfaceelements, and the second user interface includes the first set of userinterface elements and a second set of user interface elements.

Example 136: A first device, comprising: a processing unit configuredto: play media content on the first device, wherein the media content isincluded in a user interface for display on a display device; determinewhether one or more presence criteria are satisfied, wherein the one ormore presence criteria include a criterion that is satisfied while afirst person is present at the display device; and in accordance with adetermination that the one or more presence criteria are not satisfied,modify the playing of the media content in a first manner.

Example 137: The first device of preceding example 136, wherein theprocessing unit is further configured to, in accordance with adetermination that the one or more presence criteria are satisfied,maintain the playing of the media content in the user interface.

Example 138: The first device of any of examples 136-137, wherein theprocessing unit is further configured to, in accordance with thedetermination that the one or more presence criteria are not satisfied,determine a second device associated with the first person, whereinmodifying the playing of the media content in the first manner comprisestransferring the playing of the media content to the second device, thefirst device further comprising a transferring unit coupled to theprocessing unit and configured to transfer the playing of the mediacontent to the second device.

Example 139: The first device of any of examples 136-138, wherein theprocessing unit is further configured to, in accordance with adetermination that the one or more presence criteria are satisfied:determine a relative location of the first person with respect to thedisplay device, and modify the playing of the media content in a secondmanner based on the relative location of the first person, wherein thesecond manner is different from the first manner.

Example 140: The first device of any of examples 136-139, wherein theprocessing unit is further configured to, in accordance with adetermination that the one or more presence criteria are satisfied:determine a distance of the first person from the display device, andmodify the playing of the media content in a second manner based on thedistance of the first person from the display device, wherein the secondmanner is different from the first manner.

Example 141: The first device of any of examples 136-140, wherein theone or more presence criteria include a criterion that is satisfied whenthe first person is oriented towards the display device.

Example 142: The first device of any of examples 136-141, wherein theone or more presence criteria include a criterion that is satisfied whenthe first person's gaze is directed at the display device.

Example 143: The first device of any of examples 136-142, wherein theprocessing unit is further configured to: while displaying the mediacontent to a single person, detect that two or more people are presentat the display device; and in accordance with a determination that twoor more people are present at the display device, modify the playing ofthe media content in a second manner by increasing a visibility of themedia content in the user interface.

Example 144: The first device of any of examples 136-142, wherein theprocessing unit is further configured to: while displaying the mediacontent to a single person, detect that two or more people are presentat the display device; and in accordance with a determination that twoor more people are present at the display device: determine whether themedia content is designated as private media content, and in accordancewith a determination that the media content is designated as privatemedia content, modify the playing of the media content in a secondmanner by reducing a visibility of the media content in the userinterface.

Example 145: The first device of preceding example 144, wherein theprocessing unit is further configured to: in accordance with adetermination that the media content is not designated as private mediacontent, modify the playing of the media content in a third manner byincreasing the visibility of the media content in the user interface.

Example 146: The first device of any of examples 136-145, wherein theprocessing unit is further configured to, in accordance with adetermination that the one or more presence criteria are satisfied:determine whether a second device is present at the display device withthe first person, and whether a specified type of activity is beingperformed on the second device; and in accordance with a determinationthat the second device is present at the display device with the firstperson and that the specified type of activity is being performed on thesecond device, perform an action on the first device based on theactivity being performed on the second device.

Example 147: The first device of any of examples 136-146, whereindetermining whether the one or more presence criteria are satisfied isbased on information received from one or more optical sensors, thefirst device further comprising a receiving unit coupled to theprocessing unit and configured to receive the information from the oneor more optical sensors.

Example 148: The first device of any of examples 136-147, wherein: at afirst point in time, the one or more presence criteria are notsatisfied, and the playing of the media content is modified in the firstmanner, and the processing unit is further configured to: at a secondpoint in time, after the modification of the playing of the mediacontent in the first manner, determine that the one or more presencecriteria are satisfied; and in response to determining that the one ormore presence criteria are satisfied, reverse the modification of theplaying of the media content in the first manner.

This disclosure further includes subject matter directed to thefollowing examples:

Example 1: A method comprising: at an electronic device with one or moreprocessors and memory: generating a user interface for display on adisplay device; receiving a first image including a face of a firstperson; identifying the face of the first person from the first image;and performing an action on the electronic device in accordance with theidentification of the face of the first person.

Example 2: The method of preceding example 1, wherein the user interfaceincludes a second image that includes a face of a second person, theface of the second person being different from the face of the firstperson, the method further comprising: identifying the face of thesecond person from the second image, wherein performing the action onthe electronic device in accordance with the identification of the faceof the first person comprises replacing the face of the second person inthe second image with the face of the first person in the first image.

Example 3: The method of any of examples 1-2, further comprising:determining an orientation of the face of the first person in the firstimage with respect to the display device, wherein performing the actionon the electronic device in accordance with the identification of theface of the first person comprises aligning the user interface fordisplay on the display device with the orientation of the face of thefirst person in the first image.

Example 4: The method of preceding example 3, wherein aligning the userinterface with the orientation of the face of the first person comprisesforgoing aligning the user interface based on an orientation of thedevice detected by one or more sensors of the device.

Example 5: The method of any of examples 1-4, wherein: the userinterface includes a content item being played, and performing theaction on the electronic device in accordance with the identification ofthe face of the first person comprises associating the playing of thecontent item with the first person.

Example 6: The method of preceding example 5, further comprising:identifying, based on the face of the first person in the first image, afirst profile on the electronic device that is associated with the faceof the first person, wherein associating the playing of the content itemwith the first person comprises associating the playing of the contentitem with the first profile.

Example 7: The method of preceding example 6, wherein associating theplaying of the content item with the first profile comprises updating aplay count of the content item in the first profile.

Example 8: The method of any of examples 5-7, wherein the first imagefurther includes a face of a second person, the method furthercomprising: identifying the face of the second person from the firstimage; identifying, based on the face of the second person in the firstimage, a second profile on the electronic device that is associated withthe face of the second person; and associating the playing of thecontent item with the second profile.

Example 9: The method of any of examples 5-8, further comprising:receiving a second image including the face of the first person;identifying the face of the first person from the second image; inresponse to identifying the face of the first person from the secondimage, continuing to associate the playing of the content item with thefirst person; receiving a third image including the face of the firstperson and a face of a second person, wherein the third image wascaptured after the second image was captured; identifying the face ofthe first person and the face of the second person from the third image;and in response to identifying the face of the first person and the faceof the second person from the third image: continuing to associate theplaying of the content item with the first person, and associating theplaying of the content item with the second person.

Example 10: The method of any of examples 1-9, wherein: the userinterface includes a content item being played, and performing theaction on the electronic device in accordance with the identification ofthe face of the first person comprises making a preliminarilyassociation of the playing of the content item with the first person;and the method further comprises: receiving a second image, wherein thesecond image is received at least a first time threshold after receivingthe first image; and after receiving the second image, determiningwhether the second image includes the face of the first person: inaccordance with a determination that the second image does not includethe face of the first person, canceling the preliminary association ofthe playing of the content item with the first person; and in accordancewith a determination that the second image includes the face of thefirst person, confirming the preliminary association of the playing ofthe content item with the first person.

Example 11: An electronic device, comprising: one or more processors;memory; and one or more programs, wherein the one or more programs arestored in the memory and configured to be executed by the one or moreprocessors, the one or more programs including instructions for:generating a user interface for display on a display device; receiving afirst image including a face of a first person; identifying the face ofthe first person from the first image; and performing an action on theelectronic device in accordance with the identification of the face ofthe first person.

Example 12: A non-transitory computer readable storage medium storingone or more programs, the one or more programs comprising instructions,which when executed by an electronic device with one or more processorsand memory, cause the device to: generate a user interface for displayon a display device; receive a first image including a face of a firstperson; identify the face of the first person from the first image; andperform an action on the electronic device in accordance with theidentification of the face of the first person.

Example 13: An electronic device, comprising: one or more processors;memory; means for generating a user interface for display on a displaydevice; means for receiving a first image including a face of a firstperson; means for identifying the face of the first person from thefirst image; and means for performing an action on the electronic devicein accordance with the identification of the face of the first person.

Example 14: An information processing apparatus for use in an electronicdevice with one or more processors and memory, comprising: means forgenerating a user interface for display on a display device; means forreceiving a first image including a face of a first person; means foridentifying the face of the first person from the first image; and meansfor performing an action on the electronic device in accordance with theidentification of the face of the first person.

Example 15: An electronic device, comprising: one or more processors;memory; and one or more programs, wherein the one or more programs arestored in the memory and configured to be executed by the one or moreprocessors, the one or more programs including instructions forperforming any of the methods of examples 1-10.

Example 16: A non-transitory computer readable storage medium storingone or more programs, the one or more programs comprising instructions,which when executed by an electronic device with one or more processorsand memory, cause the device to perform any of the methods of examples1-10.

Example 17: An electronic device, comprising: one or more processors;memory; and means for performing any of the methods of examples 1-10.

Example 18: An information processing apparatus for use in an electronicdevice with one or more processors and memory, comprising: means forperforming any of the method of examples 1-10.

Example 19: A method comprising: at an electronic device with one ormore processors and memory: generating a user interface for display on adisplay device, wherein the user interface includes a first userinterface element; determining a gaze location in the user interface,wherein the gaze location comprises a location in the user interface towhich a person's gaze is directed; determining whether the gaze locationin the user interface coincides with the first user interface element;and in accordance with a determination that the gaze location coincideswith the first user interface element, performing an action with respectto the first user interface element.

Example 20: The method of preceding example 19, wherein: the first userinterface element comprises a user interface element requestingattention, and performing the action with respect to the first userinterface element comprises acknowledging the request for attention inresponse to determining that the gaze location coincides with the firstuser interface element.

Example 21: The method of any of examples 19-20, wherein: the userinterface further includes a second user interface element, andperforming the action with respect to the first user interface elementcomprises emphasizing the first user interface element relative to thesecond user interface element.

Example 22: The method of any of examples 19-21, further comprising:determining one or more portions of the display device that are notcoincident with the gaze location and the first user interface element,wherein performing the action with respect to the first user interfaceelement comprises adjusting an output of the one or more portions of thedisplay device that are not coincident with the gaze location.

Example 23: The method of any of examples 19-22, wherein performing theaction with respect to the first user interface element comprisesselecting the first user interface element.

Example 24: The method of any of examples 19-23, wherein the first userinterface element comprises a plurality of moving items, the methodfurther comprising: determining a first item of the plurality of movingitems with which the gaze location coincides, wherein performing theaction with respect to the first user interface element comprisesmodifying a movement of the first item of the plurality of moving itemssuch that the first item is visible in the user interface.

Example 25: The method of preceding example 24, wherein the first userinterface element comprises an autoscrolling list, and the plurality ofmoving items comprise elements of the autoscrolling list.

Example 26: The method of any of examples 24-25, wherein the first userinterface element comprises an image catalog, and the plurality ofmoving items comprise images in the image catalog.

Example 27: The method of any of examples 24-26, wherein the first userinterface element comprises a content bar, and the plurality of movingitems comprise representations of content items accessible from thecontent bar.

Example 28: The method of any of examples 19-27, wherein: the first userinterface element comprises content, and performing the action withrespect to the first user interface element comprises designating thecontent in the first user interface element as being read.

Example 29: The method of any of examples 19-28, wherein performing theaction with respect to the first user interface element comprisesdesignating the first user interface element as an active user interfaceelement, the method further comprising: receiving a first input from aninput device for interacting with an active user interface element; andin response to the first input, interacting with the first userinterface element in accordance with the first input.

Example 30: The method of preceding example 29, wherein: the userinterface further includes a second user interface element, a locationof a cursor associated with the input device is coincident with thesecond user interface element, and designating the first user interfaceelement as the active user interface element comprises ignoring thelocation of the cursor in designating the first user interface elementas the active user interface element.

Example 31: An electronic device, comprising: one or more processors;memory; and one or more programs, wherein the one or more programs arestored in the memory and configured to be executed by the one or moreprocessors, the one or more programs including instructions for:generating a user interface for display on a display device, wherein theuser interface includes a first user interface element; determining agaze location in the user interface, wherein the gaze location comprisesa location in the user interface to which a person's gaze is directed;determining whether the gaze location in the user interface coincideswith the first user interface element; and in accordance with adetermination that the gaze location coincides with the first userinterface element, performing an action with respect to the first userinterface element.

Example 32: A non-transitory computer readable storage medium storingone or more programs, the one or more programs comprising instructions,which when executed by an electronic device with one or more processorsand memory, cause the device to: generate a user interface for displayon a display device, wherein the user interface includes a first userinterface element; determine a gaze location in the user interface,wherein the gaze location comprises a location in the user interface towhich a person's gaze is directed; determine whether the gaze locationin the user interface coincides with the first user interface element;and in accordance with a determination that the gaze location coincideswith the first user interface element, perform an action with respect tothe first user interface element.

Example 33: An electronic device, comprising: one or more processors;memory; means for generating a user interface for display on a displaydevice, wherein the user interface includes a first user interfaceelement; means for determining a gaze location in the user interface,wherein the gaze location comprises a location in the user interface towhich a person's gaze is directed; means for determining whether thegaze location in the user interface coincides with the first userinterface element; and means for, in accordance with a determinationthat the gaze location coincides with the first user interface element,performing an action with respect to the first user interface element.

Example 34: An information processing apparatus for use in an electronicdevice with one or more processors and memory, comprising: means forgenerating a user interface for display on a display device, wherein theuser interface includes a first user interface element; means fordetermining a gaze location in the user interface, wherein the gazelocation comprises a location in the user interface to which a person'sgaze is directed; means for determining whether the gaze location in theuser interface coincides with the first user interface element; andmeans for, in accordance with a determination that the gaze locationcoincides with the first user interface element, performing an actionwith respect to the first user interface element.

Example 35: An electronic device, comprising: one or more processors;memory; and one or more programs, wherein the one or more programs arestored in the memory and configured to be executed by the one or moreprocessors, the one or more programs including instructions forperforming any of the methods of examples 19-30.

Example 36: A non-transitory computer readable storage medium storingone or more programs, the one or more programs comprising instructions,which when executed by an electronic device with one or more processorsand memory, cause the device to perform any of the methods of examples19-30.

Example 37: An electronic device, comprising: one or more processors;memory; and means for performing any of the methods of examples 19-30.

Example 38: An information processing apparatus for use in an electronicdevice with one or more processors and memory, comprising: means forperforming any of the method of examples 19-30.

Example 39: A method comprising: at an electronic device with one ormore processors and memory: generating a first sound for playback on asound-playback device; receiving an output from a microphone on asound-receiving device; determining whether the sound-receiving deviceis present at the sound-playback device based on a plurality of presencecriteria, including: sound-detection criteria that is satisfied when theoutput from the microphone on the sound-receiving device corresponds tothe first sound for playback on the sound-playback device; andverification criteria that is satisfied when a presence of thesound-receiving device at the sound-playback device is verified using aquantity other than the first sound; and in accordance with adetermination that the plurality of presence criteria are satisfied,designating a user of the sound-receiving device as being present at thesound-playback device.

Example 40: The method of preceding example 39, wherein designating theuser of the sound-receiving device as being present at thesound-playback device comprises determining a user associated with thesound-receiving device, and designating the user as being present at thesound-playback device.

Example 41: The method of any of examples 39-40, further comprisingprior to generating the first sound for playback on the sound-playbackdevice, transmitting a wake command to the sound-receiving device forturning on the microphone on the sound-receiving device.

Example 42: The method of any of examples 39-41, wherein theverification criteria include a criterion that is satisfied when a faceof the user of the sound-receiving device is recognized at thesound-playback device.

Example 43: The method of any of examples 39-42, wherein theverification criteria include a criterion that is satisfied when thesound-receiving device has sensed motion during a most recent timeperiod.

Example 44: The method of any of examples 39-43, wherein theverification criteria include a criterion that is satisfied when thesound-receiving device resides on a same wireless network as thesound-playback device.

Example 45: The method of any of examples 39-44, further comprising:detecting motion of the sound-receiving device; and in accordance with adetermination that the motion exceeds a predefined threshold,re-evaluating the plurality of presence criteria to determine whetherthe sound-receiving device remains at the sound-playback device.

Example 46: The method of any of examples 39-45, wherein: thesound-playback device is configured to transition from a first state toa second state in response to one or more transition criteria beingsatisfied at the sound-playback device, the method further comprising:in accordance with the determination that the plurality of presencecriteria are satisfied, forgoing transitioning from the first state ofthe sound-playback device to the second state of the sound-playbackdevice in response to the one or more transition criteria beingsatisfied at the sound-playback device.

Example 47: An electronic device, comprising: one or more processors;memory; and one or more programs, wherein the one or more programs arestored in the memory and configured to be executed by the one or moreprocessors, the one or more programs including instructions for:generating a first sound for playback on a sound-playback device;receiving an output from a microphone on a sound-receiving device;determining whether the sound-receiving device is present at thesound-playback device based on a plurality of presence criteria,including: sound-detection criteria that is satisfied when the outputfrom the microphone on the sound-receiving device corresponds to thefirst sound for playback on the sound-playback device; and verificationcriteria that is satisfied when a presence of the sound-receiving deviceat the sound-playback device is verified using a quantity other than thefirst sound; and in accordance with a determination that the pluralityof presence criteria are satisfied, designating a user of thesound-receiving device as being present at the sound-playback device.

Example 48: A non-transitory computer readable storage medium storingone or more programs, the one or more programs comprising instructions,which when executed by an electronic device with one or more processorsand memory, cause the device to: generate a first sound for playback ona sound-playback device; receive an output from a microphone on asound-receiving device; determine whether the sound-receiving device ispresent at the sound-playback device based on a plurality of presencecriteria, including: sound-detection criteria that is satisfied when theoutput from the microphone on the sound-receiving device corresponds tothe first sound for playback on the sound-playback device; andverification criteria that is satisfied when a resence of thesound-receiving device at the sound-playback device is verified using aquantity other than the first sound; and in accordance with adetermination that the plurality of presence criteria are satisfied,designate a user of the sound-receiving device as being present at thesound-playback device.

Example 49: An electronic device, comprising: one or more processors;memory; means for generating a first sound for playback on asound-playback device; means for receiving an output from a microphoneon a sound-receiving device; means for determining whether thesound-receiving device is present at the sound-playback device based ona plurality of presence criteria, including: sound-detection criteriathat is satisfied when the output from the microphone on thesound-receiving device corresponds to the first sound for playback onthe sound-playback device; and verification criteria that is satisfiedwhen a presence of the sound-receiving device at the sound-playbackdevice is verified using a quantity other than the first sound; andmeans for, in accordance with a determination that the plurality ofpresence criteria are satisfied, designating a user of thesound-receiving device as being present at the sound-playback device.

Example 50: An information processing apparatus for use in an electronicdevice with one or more processors and memory, comprising: means forgenerating a first sound for playback on a sound-playback device; meansfor receiving an output from a microphone on a sound-receiving device;means for determining whether the sound-receiving device is present atthe sound-playback device based on a plurality of presence criteria,including: sound-detection criteria that is satisfied when the outputfrom the microphone on the sound-receiving device corresponds to thefirst sound for playback on the sound-playback device; and verificationcriteria that is satisfied when a presence of the sound-receiving deviceat the sound-playback device is verified using a quantity other than thefirst sound; and means for, in accordance with a determination that theplurality of presence criteria are satisfied, designating a user of thesound-receiving device as being present at the sound-playback device.

Example 51: An electronic device, comprising: one or more processors;memory; and one or more programs, wherein the one or more programs arestored in the memory and configured to be executed by the one or moreprocessors, the one or more programs including instructions forperforming any of the methods of examples 39-46.

Example 52: A non-transitory computer readable storage medium storingone or more programs, the one or more programs comprising instructions,which when executed by an electronic device with one or more processorsand memory, cause the device to perform any of the methods of examples39-46.

Example 53: An electronic device, comprising: one or more processors;memory; and means for performing any of the methods of examples 39-46.

Example 54: An information processing apparatus for use in an electronicdevice with one or more processors and memory, comprising: means forperforming any of the method of examples 39-46.

Example 55: A method comprising: at an electronic device with one ormore processors and memory: determining whether a person is further thana threshold distance from a display device or closer than the thresholddistance from the display device; in accordance with a determinationthat the person is further than the threshold distance from the displaydevice, providing a first user interface for display on the displaydevice; and in accordance with a determination that the person is closerthan the threshold distance from the display device, providing a seconduser interface for display on the display device, wherein the seconduser interface is different than the first user interface.

Example 56: The method of preceding example 55, wherein determiningwhether the person is further than the threshold distance from thedisplay device or closer than the threshold distance from the displaydevice is based on information received from one or more opticalsensors.

Example 57: The method of preceding example 56, further comprisingidentifying a face of the person based on the information received fromthe one or more optical sensors, wherein determining whether the personis further than the threshold distance from the display device or closerthan the threshold distance from the display device is based on a sizeof the face of the person.

Example 58: The method of any of examples 55-57, wherein the person isdetermined to be further than the threshold distance from the displaydevice, the method further comprising: while the first user interface isdisplayed on the display device, detecting movement of the person to adistance closer than the threshold distance from the display device; andin response to detecting the movement of the person to the distancecloser than the threshold distance, providing the second user interfacefor display on the display device.

Example 59: The method of any of examples 55-57, wherein the person isdetermined to be closer than the threshold distance from the displaydevice, the method further comprising: while the second user interfaceis displayed on the display device, detecting movement of the person toa distance further than the threshold distance from the display device;and in response to detecting the movement of the person to the distancefurther than the threshold distance, providing the first user interfacefor display on the display device.

Example 60: The method of any of examples 55-58, wherein: the person isfurther than the threshold distance from the display device, and thefirst user interface is provided for display on the displace device, thefirst user interface including a first user interface element and asecond user interface element, the method further comprising: while thefirst user interface is displayed on the display device: determining agaze location in the first user interface, wherein the gaze locationcomprises a location in the first user interface to which the person'sgaze is directed; detecting movement of the person to a distance closerthan the threshold distance from the display device; and in response todetecting the movement of the person, providing the second userinterface for display on the display device, the second user interfaceincluding information based on the gaze location.

Example 61: The method of preceding example 60, further comprising:determining whether the gaze location coincides with the first userinterface element or the second user interface element in the first userinterface, wherein in response to detecting the movement of the personto the distance closer than the threshold distance from the displaydevice: in accordance with a determination that the gaze locationcoincides with the first user interface element, providing the seconduser interface for display on the display device, the second userinterface including information associated with the first user interfaceelement, and in accordance with a determination that the gaze locationcoincides with the second user interface element, providing the seconduser interface for display on the display device, the second userinterface including information associated with the second userinterface element.

Example 62: The method of any of examples 55-57 and 59, wherein: theperson is closer than the threshold distance from the display device,and the person is logged into one or more areas on the electronicdevice, the method further comprising: detecting movement of the personto a distance further than the threshold distance from the displaydevice; and in response to detecting the movement of the person to thedistance further than the threshold distance from the display device,logging out of the one or more areas on the electronic device.

Example 63: The method of any of examples 55-62, wherein the first userinterface includes a user interface element at a first size, and thesecond user interface includes the user interface element at a secondsize, smaller than the first size.

Example 64: The method of any of examples 55-63, wherein the first userinterface includes summary information about a first quantity, and thesecond user interface includes detailed information about the firstquantity.

Example 65: The method of any of examples 55-64, wherein the first userinterface includes a first set of user interface elements, and thesecond user interface includes the first set of user interface elementsand a second set of user interface elements.

Example 66: An electronic device, comprising: one or more processors;memory; and one or more programs, wherein the one or more programs arestored in the memory and configured to be executed by the one or moreprocessors, the one or more programs including instructions for:determining whether a person is further than a threshold distance from adisplay device or closer than the threshold distance from the displaydevice; in accordance with a determination that the person is furtherthan the threshold distance from the display device, providing a firstuser interface for display on the display device; and in accordance witha determination that the person is closer than the threshold distancefrom the display device, providing a second user interface for displayon the display device, wherein the second user interface is differentthan the first user interface.

Example 67: A non-transitory computer readable storage medium storingone or more programs, the one or more programs comprising instructions,which when executed by an electronic device with one or more processorsand memory, cause the device to: determine whether a person is furtherthan a threshold distance from a display device or closer than thethreshold distance from the display device; in accordance with adetermination that the person is further than the threshold distancefrom the display device, provide a first user interface for display onthe display device; and in accordance with a determination that theperson is closer than the threshold distance from the display device,provide a second user interface for display on the display device,wherein the second user interface is different than the first userinterface.

Example 68: An electronic device, comprising: one or more processors;memory; means for determining whether a person is further than athreshold distance from a display device or closer than the thresholddistance from the display device; means for, in accordance with adetermination that the person is further than the threshold distancefrom the display device, providing a first user interface for display onthe display device; and means for, in accordance with a determinationthat the person is closer than the threshold distance from the displaydevice, providing a second user interface for display on the displaydevice, wherein the second user interface is different than the firstuser interface.

Example 69: An information processing apparatus for use in an electronicdevice with one or more processors and memory, comprising: means fordetermining whether a person is further than a threshold distance from adisplay device or closer than the threshold distance from the displaydevice; means for, in accordance with a determination that the person isfurther than the threshold distance from the display device, providing afirst user interface for display on the display device; and means for,in accordance with a determination that the person is closer than thethreshold distance from the display device, providing a second userinterface for display on the display device, wherein the second userinterface is different than the first user interface.

Example 70: An electronic device, comprising: one or more processors;memory; and one or more programs, wherein the one or more programs arestored in the memory and configured to be executed by the one or moreprocessors, the one or more programs including instructions forperforming any of the methods of examples 55-65.

Example 71: A non-transitory computer readable storage medium storingone or more programs, the one or more programs comprising instructions,which when executed by an electronic device with one or more processorsand memory, cause the device to perform any of the methods of examples55-65.

Example 72: An electronic device, comprising: one or more processors;memory; and means for performing any of the methods of examples 55-65.

Example 73: An information processing apparatus for use in an electronicdevice with one or more processors and memory, comprising: means forperforming any of the method of examples 55-65.

Example 74: A method comprising: at a first device with one or moreprocessors and memory: playing media content on the first device,wherein the media content is included in a user interface for display ona display device; determining whether one or more presence criteria aresatisfied, wherein the one or more presence criteria include a criterionthat is satisfied while a first person is present at the display device;and in accordance with a determination that the one or more presencecriteria are not satisfied, modifying the playing of the media contentin a first manner.

Example 75: The method of preceding example 74, further comprising inaccordance with a determination that the one or more presence criteriaare satisfied, maintaining the playing of the media content in the userinterface.

Example 76: The method of any of examples 74-75, further comprising: inaccordance with the determination that the one or more presence criteriaare not satisfied, determining a second device associated with the firstperson, wherein modifying the playing of the media content in the firstmanner comprises transferring the playing of the media content to thesecond device.

Example 77: The method of any of examples 74-76, further comprising inaccordance with a determination that the one or more presence criteriaare satisfied: determining a relative location of the first person withrespect to the display device, and modifying the playing of the mediacontent in a second manner based on the relative location of the firstperson, wherein the second manner is different from the first manner.

Example 78: The method of any of examples 74-77, further comprising inaccordance with a determination that the one or more presence criteriaare satisfied: determining a distance of the first person from thedisplay device, and modifying the playing of the media content in asecond manner based on the distance of the first person from the displaydevice, wherein the second manner is different from the first manner.

Example 79: The method of any of examples 74-78, wherein the one or morepresence criteria include a criterion that is satisfied when the firstperson is oriented towards the display device.

Example 80: The method of any of examples 74-79, wherein the one or morepresence criteria include a criterion that is satisfied when the firstperson's gaze is directed at the display device.

Example 81: The method of any of examples 74-80, further comprising:while displaying the media content to a single person, detecting thattwo or more people are present at the display device; and in accordancewith a determination that two or more people are present at the displaydevice, modifying the playing of the media content in a second manner byincreasing a visibility of the media content in the user interface.

Example 82: The method of any of examples 74-80, further comprising:while displaying the media content to a single person, detecting thattwo or more people are present at the display device; and in accordancewith a determination that two or more people are present at the displaydevice: determining whether the media content is designated as privatemedia content, and in accordance with a determination that the mediacontent is designated as private media content, modifying the playing ofthe media content in a second manner by reducing a visibility of themedia content in the user interface.

Example 83: The method of preceding example 82, further comprising: inaccordance with a determination that the media content is not designatedas private media content, modifying the playing of the media content ina third manner by increasing the visibility of the media content in theuser interface.

Example 84: The method of any of examples 74-83, further comprising inaccordance with a determination that the one or more presence criteriaare satisfied: determining whether a second device is present at thedisplay device with the first person, and whether a specified type ofactivity is being performed on the second device; and in accordance witha determination that the second device is present at the display devicewith the first person and that the specified type of activity is beingperformed on the second device, performing an action on the first devicebased on the activity being performed on the second device.

Example 85: The method of any of examples 74-84, wherein determiningwhether the one or more presence criteria are satisfied is based oninformation received from one or more optical sensors.

Example 86: The method of any of examples 74-85, wherein: at a firstpoint in time, the one or more presence criteria are not satisfied, andthe playing of the media content is modified in the first manner, themethod further comprising: at a second point in time, after themodification of the playing of the media content in the first manner,determining that the one or more presence criteria are satisfied; and inresponse to determining that the one or more presence criteria aresatisfied, reversing the modification of the playing of the mediacontent in the first manner.

Example 87: A first electronic device, comprising: one or moreprocessors; memory; and one or more programs, wherein the one or moreprograms are stored in the memory and configured to be executed by theone or more processors, the one or more programs including instructionsfor: playing media content on the first device, wherein the mediacontent is included in a user interface for display on a display device;determining whether one or more presence criteria are satisfied, whereinthe one or more presence criteria include a criterion that is satisfiedwhile a first person is present at the display device; and in accordancewith a determination that the one or more presence criteria are notsatisfied, modifying the playing of the media content in a first manner.

Example 88: A non-transitory computer readable storage medium storingone or more programs, the one or more programs comprising instructions,which when executed by a first electronic device with one or moreprocessors and memory, cause the first device to: play media content onthe first device, wherein the media content is included in a userinterface for display on a display device; determine whether one or morepresence criteria are satisfied, wherein the one or more presencecriteria include a criterion that is satisfied while a first person ispresent at the display device; and in accordance with a determinationthat the one or more presence criteria are not satisfied, modify theplaying of the media content in a first manner.

Example 89: A first electronic device, comprising: one or moreprocessors; memory; means for playing media content on the first device,wherein the media content is included in a user interface for display ona display device; means for determining whether one or more presencecriteria are satisfied, wherein the one or more presence criteriainclude a criterion that is satisfied while a first person is present atthe display device; and means for, in accordance with a determinationthat the one or more presence criteria are not satisfied, modifying theplaying of the media content in a first manner.

Example 90: An information processing apparatus for use in a firstelectronic device with one or more processors and memory, comprising:means for playing media content on the first device, wherein the mediacontent is included in a user interface for display on a display device;means for determining whether one or more presence criteria aresatisfied, wherein the one or more presence criteria include a criterionthat is satisfied while a first person is present at the display device;and means for, in accordance with a determination that the one or morepresence criteria are not satisfied, modifying the playing of the mediacontent in a first manner.

Example 91: An electronic device, comprising: one or more processors;memory; and one or more programs, wherein the one or more programs arestored in the memory and configured to be executed by the one or moreprocessors, the one or more programs including instructions forperforming any of the methods of examples 74-86.

Example 92: A non-transitory computer readable storage medium storingone or more programs, the one or more programs comprising instructions,which when executed by an electronic device with one or more processorsand memory, cause the device to perform any of the methods of examples74-86.

Example 93: An electronic device, comprising: one or more processors;memory; and means for performing any of the methods of examples 74-86.

Example 94: An information processing apparatus for use in an electronicdevice with one or more processors and memory, comprising: means forperforming any of the method of examples 74-86.

Example 95: An electronic device, comprising: a receiving unitconfigured to receive a first image including a face of a first person;and a processing unit coupled to the receiving unit, the processing unitconfigured to: generate a user interface for display on a displaydevice; identify the face of the first person from the first image; andperform an action on the electronic device in accordance with theidentification of the face of the first person.

Example 96: The electronic device of preceding example 95, wherein theuser interface includes a second image that includes a face of a secondperson, the face of the second person being different from the face ofthe first person, and the processing unit is further configured to:identify the face of the second person from the second image, whereinperforming the action on the electronic device in accordance with theidentification of the face of the first person comprises replacing theface of the second person in the second image with the face of the firstperson in the first image.

Example 97: The electronic device of any of examples 95-96, wherein theprocessing unit is further configured to: determine an orientation ofthe face of the first person in the first image with respect to thedisplay device, wherein performing the action on the electronic devicein accordance with the identification of the face of the first personcomprises aligning the user interface for display on the display devicewith the orientation of the face of the first person in the first image.

Example 98: The electronic device of preceding example 97, whereinaligning the user interface with the orientation of the face of thefirst person comprises forgoing aligning the user interface based on anorientation of the device detected by one or more sensor units of thedevice, the sensor units being coupled to the processing unit and thereceiving unit.

Example 99: The electronic device of any of examples 95-98, wherein: theuser interface includes a content item being played, and performing theaction on the electronic device in accordance with the identification ofthe face of the first person comprises associating the playing of thecontent item with the first person.

Example 100: The electronic device of preceding example 99, wherein theprocessing unit is further configured to: identify, based on the face ofthe first person in the first image, a first profile on the electronicdevice that is associated with the face of the first person, whereinassociating the playing of the content item with the first personcomprises associating the playing of the content item with the firstprofile.

Example 101: The electronic device of preceding example 100, whereinassociating the playing of the content item with the first profilecomprises updating a play count of the content item in the firstprofile.

Example 102: The electronic device of any of examples 99-101, whereinthe first image further includes a face of a second person, and theprocessing unit is further configured to: identify the face of thesecond person from the first image; identify, based on the face of thesecond person in the first image, a second profile on the electronicdevice that is associated with the face of the second person; andassociate the playing of the content item with the second profile.

Example 103: The electronic device of any of examples 99-102, wherein:the receiving unit is further configured to receive a second imageincluding the face of the first person, and receive a third imageincluding the face of the first person and a face of a second person,wherein the third image is captured after the second image was captured,and the processing unit is further configured to: identify the face ofthe first person from the second image; in response to identifying theface of the first person from the second image, continue to associatethe playing of the content item with the first person; identify the faceof the first person and the face of the second person from the thirdimage; and in response to identifying the face of the first person andthe face of the second person from the third image: continue toassociate the playing of the content item with the first person, andassociate the playing of the content item with the second person.

Example 104: The electronic device of any of examples 95-103, wherein:the user interface includes a content item being played, performing theaction on the electronic device in accordance with the identification ofthe face of the first person comprises making a preliminarilyassociation of the playing of the content item with the first person,the receiving unit is further configured to receive a second image,wherein the second image is received at least a first time thresholdafter receiving the first image, and the processing unit is furtherconfigured to: after receiving the second image, determine whether thesecond image includes the face of the first person: in accordance with adetermination that the second image does not include the face of thefirst person, cancel the preliminary association of the playing of thecontent item with the first person; and in accordance with adetermination that the second image includes the face of the firstperson, confirm the preliminary association of the playing of thecontent item with the first person.

Example 105: An electronic device, comprising: a processing unitconfigured to: generate a user interface for display on a displaydevice, wherein the user interface includes a first user interfaceelement; determine a gaze location in the user interface, wherein thegaze location comprises a location in the user interface to which aperson's gaze is directed; determine whether the gaze location in theuser interface coincides with the first user interface element; and inaccordance with a determination that the gaze location coincides withthe first user interface element, perform an action with respect to thefirst user interface element.

Example 106: The electronic device of preceding example 105, wherein:the first user interface element comprises a user interface elementrequesting attention, and performing the action with respect to thefirst user interface element comprises acknowledging the request forattention in response to determining that the gaze location coincideswith the first user interface element.

Example 107: The electronic device of any of examples 105-106, wherein:the user interface further includes a second user interface element, andperforming the action with respect to the first user interface elementcomprises emphasizing the first user interface element relative to thesecond user interface element.

Example 108: The electronic device of any of examples 105-107, whereinthe processing unit is further configured to: determine one or moreportions of the display device that are not coincident with the gazelocation and the first user interface element, wherein performing theaction with respect to the first user interface element comprisesadjusting an output of the one or more portions of the display devicethat are not coincident with the gaze location.

Example 109: The electronic device of any of examples 105-108, whereinperforming the action with respect to the first user interface elementcomprises selecting the first user interface element.

Example 110: The electronic device of any of examples 105-109, whereinthe first user interface element comprises a plurality of moving items,and the processing unit is further configured to: determine a first itemof the plurality of moving items with which the gaze location coincides,wherein performing the action with respect to the first user interfaceelement comprises modifying a movement of the first item of theplurality of moving items such that the first item is visible in theuser interface.

Example 111: The electronic device of preceding example 110, wherein thefirst user interface element comprises an autoscrolling list, and theplurality of moving items comprise elements of the autoscrolling list.

Example 112: The electronic device of any of examples 110-111, whereinthe first user interface element comprises an image catalog, and theplurality of moving items comprise images in the image catalog.

Example 113: The electronic device of any of examples 110-112, whereinthe first user interface element comprises a content bar, and theplurality of moving items comprise representations of content itemsaccessible from the content bar.

Example 114: The electronic device of any of examples 105-113, wherein:the first user interface element comprises content, and performing theaction with respect to the first user interface element comprisesdesignating the content in the first user interface element as beingread.

Example 115: The electronic device of any of examples 105-114, whereinperforming the action with respect to the first user interface elementcomprises designating the first user interface element as an active userinterface element, the electronic device further comprising: a receivingunit configured to receive a first input from an input unit forinteracting with an active user interface element, the receiving unitbeing coupled to the processing unit, wherein the processing unit isfurther configured to: in response to the first input, interact with thefirst user interface element in accordance with the first input.

Example 116: The electronic device of preceding example 115, wherein:the user interface further includes a second user interface element, alocation of a cursor associated with the input device is coincident withthe second user interface element, and designating the first userinterface element as the active user interface element comprisesignoring the location of the cursor in designating the first userinterface element as the active user interface element.

Example 117: An electronic device, comprising: a receiving unitconfigured to receive an output from a microphone on a sound-receivingdevice; and a processing unit coupled to the receiving unit andconfigured to: generate a first sound for playback on a sound-playbackdevice; determine whether the sound-receiving device is present at thesound-playback device based on a plurality of presence criteria,including: sound-detection criteria that is satisfied when the outputfrom the microphone on the sound-receiving device corresponds to thefirst sound for playback on the sound-playback device; and verificationcriteria that is satisfied when a presence of the sound-receiving deviceat the sound-playback device is verified using a quantity other than thefirst sound; and in accordance with a determination that the pluralityof presence criteria are satisfied, designate a user of thesound-receiving device as being present at the sound-playback device.

Example 118: The electronic device of preceding example 117, whereindesignating the user of the sound-receiving device as being present atthe sound-playback device comprises determining a user associated withthe sound-receiving device, and designating the user as being present atthe sound-playback device.

Example 119: The electronic device of any of examples 117-118, furthercomprising a transmitting unit coupled to the processing unit and thereceiving unit, and configured to, prior to the processing unitgenerating the first sound for playback on the sound-playback device,transmit a wake command to the sound-receiving device for turning on themicrophone on the sound-receiving device.

Example 120: The electronic device of any of examples 117-119, whereinthe verification criteria include a criterion that is satisfied when aface of the user of the sound-receiving device is recognized at thesound-playback device.

Example 121: The electronic device of any of examples 117-120, whereinthe verification criteria include a criterion that is satisfied when thesound-receiving device has sensed motion during a most recent timeperiod.

Example 122: The electronic device of any of examples 117-121, whereinthe verification criteria include a criterion that is satisfied when thesound-receiving device resides on a same wireless network as thesound-playback device.

Example 123: The electronic device of any of examples 117-122, whereinthe processing unit is further configured to: detect motion of thesound-receiving device; and in accordance with a determination that themotion exceeds a predefined threshold, re-evaluate the plurality ofpresence criteria to determine whether the sound-receiving deviceremains at the sound-playback device.

Example 124: The electronic device of any of examples 117-123, wherein:the sound-playback device is configured to transition from a first stateto a second state in response to one or more transition criteria beingsatisfied at the sound-playback device, and the processing unit isfurther configured to: in accordance with the determination that theplurality of presence criteria are satisfied, forgo transitioning fromthe first state of the sound-playback device to the second state of thesound-playback device in response to the one or more transition criteriabeing satisfied at the sound-playback device.

Example 125: An electronic device, comprising: a processing unitconfigured to: determine whether a person is further than a thresholddistance from a display device or closer than the threshold distancefrom the display device; in accordance with a determination that theperson is further than the threshold distance from the display device,provide a first user interface for display on the display device; and inaccordance with a determination that the person is closer than thethreshold distance from the display device, provide a second userinterface for display on the display device, wherein the second userinterface is different than the first user interface.

Example 126: The electronic device of preceding example 125, whereindetermining whether the person is further than the threshold distancefrom the display device or closer than the threshold distance from thedisplay device is based on information received from one or more opticalsensors, the electronic device further comprising a receiving unitcoupled to the processing unit and configured to receive the informationfrom the one or more optical sensors.

Example 127: The electronic device of preceding example 126, wherein theprocessing unit is further configured to identify a face of the personbased on the information received from the one or more optical sensors,wherein determining whether the person is further than the thresholddistance from the display device or closer than the threshold distancefrom the display device is based on a size of the face of the person.

Example 128: The electronic device of any of examples 125-127, whereinthe person is determined to be further than the threshold distance fromthe display device, and the processing unit is further configured to:while the first user interface is displayed on the display device,detect movement of the person to a distance closer than the thresholddistance from the display device; and in response to detecting themovement of the person to the distance closer than the thresholddistance, provide the second user interface for display on the displaydevice.

Example 129: The electronic device of any of examples 125-127, whereinthe person is determined to be closer than the threshold distance fromthe display device, and the processing unit is further configured to:while the second user interface is displayed on the display device,detect movement of the person to a distance further than the thresholddistance from the display device; and in response to detecting themovement of the person to the distance further than the thresholddistance, provide the first user interface for display on the displaydevice.

Example 130: The electronic device of any of examples 125-128, wherein:the person is further than the threshold distance from the displaydevice, the first user interface is provided for display on the displacedevice, the first user interface including a first user interfaceelement and a second user interface element, and the processing unit isfurther configured to: while the first user interface is displayed onthe display device: determine a gaze location in the first userinterface, wherein the gaze location comprises a location in the firstuser interface to which the person's gaze is directed; detect movementof the person to a distance closer than the threshold distance from thedisplay device; and in response to detecting the movement of the person,provide the second user interface for display on the display device, thesecond user interface including information based on the gaze location.

Example 131: The electronic device of preceding example 130, wherein theprocessing unit is further configured to: determine whether the gazelocation coincides with the first user interface element or the seconduser interface element in the first user interface, and in response todetecting the movement of the person to the distance closer than thethreshold distance from the display device: in accordance with adetermination that the gaze location coincides with the first userinterface element, provide the second user interface for display on thedisplay device, the second user interface including informationassociated with the first user interface element, and in accordance witha determination that the gaze location coincides with the second userinterface element, provide the second user interface for display on thedisplay device, the second user interface including informationassociated with the second user interface element.

Example 132: The electronic device of any of examples 125-127 and 129,wherein: the person is closer than the threshold distance from thedisplay device, the person is logged into one or more areas on theelectronic device, and the processing unit is further configured to:detect movement of the person to a distance further than the thresholddistance from the display device; and in response to detecting themovement of the person to the distance further than the thresholddistance from the display device, log out of the one or more areas onthe electronic device.

Example 133: The electronic device of any of examples 125-132, whereinthe first user interface includes a user interface element at a firstsize, and the second user interface includes the user interface elementat a second size, smaller than the first size.

Example 134: The electronic device of any of examples 125-133, whereinthe first user interface includes summary information about a firstquantity, and the second user interface includes detailed informationabout the first quantity.

Example 135: The electronic device of any of examples 125-134, whereinthe first user interface includes a first set of user interfaceelements, and the second user interface includes the first set of userinterface elements and a second set of user interface elements.

Example 136: A first device, comprising: a processing unit configuredto: play media content on the first device, wherein the media content isincluded in a user interface for display on a display device; determinewhether one or more presence criteria are satisfied, wherein the one ormore presence criteria include a criterion that is satisfied while afirst person is present at the display device; and in accordance with adetermination that the one or more presence criteria are not satisfied,modify the playing of the media content in a first manner.

Example 137: The first device of preceding example 136, wherein theprocessing unit is further configured to, in accordance with adetermination that the one or more presence criteria are satisfied,maintain the playing of the media content in the user interface.

Example 138: The first device of any of examples 136-137, wherein theprocessing unit is further configured to, in accordance with thedetermination that the one or more presence criteria are not satisfied,determine a second device associated with the first person, whereinmodifying the playing of the media content in the first manner comprisestransferring the playing of the media content to the second device, thefirst device further comprising a transferring unit coupled to theprocessing unit and configured to transfer the playing of the mediacontent to the second device.

Example 139: The first device of any of examples 136-138, wherein theprocessing unit is further configured to, in accordance with adetermination that the one or more presence criteria are satisfied:determine a relative location of the first person with respect to thedisplay device, and modify the playing of the media content in a secondmanner based on the relative location of the first person, wherein thesecond manner is different from the first manner.

Example 140: The first device of any of examples 136-139, wherein theprocessing unit is further configured to, in accordance with adetermination that the one or more presence criteria are satisfied:determine a distance of the first person from the display device, andmodify the playing of the media content in a second manner based on thedistance of the first person from the display device, wherein the secondmanner is different from the first manner.

Example 141: The first device of any of examples 136-140, wherein theone or more presence criteria include a criterion that is satisfied whenthe first person is oriented towards the display device.

Example 142: The first device of any of examples 136-141, wherein theone or more presence criteria include a criterion that is satisfied whenthe first person's gaze is directed at the display device.

Example 143: The first device of any of examples 136-142, wherein theprocessing unit is further configured to: while displaying the mediacontent to a single person, detect that two or more people are presentat the display device; and in accordance with a determination that twoor more people are present at the display device, modify the playing ofthe media content in a second manner by increasing a visibility of themedia content in the user interface.

Example 144: The first device of any of examples 136-142, wherein theprocessing unit is further configured to: while displaying the mediacontent to a single person, detect that two or more people are presentat the display device; and in accordance with a determination that twoor more people are present at the display device: determine whether themedia content is designated as private media content, and in accordancewith a determination that the media content is designated as privatemedia content, modify the playing of the media content in a secondmanner by reducing a visibility of the media content in the userinterface.

Example 145: The first device of preceding example 144, wherein theprocessing unit is further configured to: in accordance with adetermination that the media content is not designated as private mediacontent, modify the playing of the media content in a third manner byincreasing the visibility of the media content in the user interface.

Example 146: The first device of any of examples 136-145, wherein theprocessing unit is further configured to, in accordance with adetermination that the one or more presence criteria are satisfied:determine whether a second device is present at the display device withthe first person, and whether a specified type of activity is beingperformed on the second device; and in accordance with a determinationthat the second device is present at the display device with the firstperson and that the specified type of activity is being performed on thesecond device, perform an action on the first device based on theactivity being performed on the second device.

Example 147: The first device of any of examples 136-146, whereindetermining whether the one or more presence criteria are satisfied isbased on information received from one or more optical sensors, thefirst device further comprising a receiving unit coupled to theprocessing unit and configured to receive the information from the oneor more optical sensors.

Example 148: The first device of any of examples 136-147, wherein: at afirst point in time, the one or more presence criteria are notsatisfied, and the playing of the media content is modified in the firstmanner, and the processing unit is further configured to: at a secondpoint in time, after the modification of the playing of the mediacontent in the first manner, determine that the one or more presencecriteria are satisfied; and in response to determining that the one ormore presence criteria are satisfied, reverse the modification of theplaying of the media content in the first manner.

The foregoing description, for purpose of explanation, has beendescribed with reference to specific embodiments. However, theillustrative discussions above are not intended to be exhaustive or tolimit the invention to the precise forms disclosed. Many modificationsand variations are possible in view of the above teachings. Theembodiments were chosen and described in order to best explain theprinciples of the invention and its practical applications, to therebyenable others skilled in the art to best use the invention and variousdescribed embodiments with various modifications as are suited to theparticular use contemplated.

1. A method comprising: at an electronic device with one or moreprocessors and memory: determining whether a person is further than athreshold distance from a display device or closer than the thresholddistance from the display device; in accordance with a determinationthat the person is further than the threshold distance from the displaydevice, providing a first user interface for display on the displaydevice; and in accordance with a determination that the person is closerthan the threshold distance from the display device, providing a seconduser interface for display on the display device, wherein the seconduser interface is different than the first user interface.
 2. The methodof claim 1, wherein determining whether the person is further than thethreshold distance from the display device or closer than the thresholddistance from the display device is based on information received fromone or more optical sensors.
 3. The method of claim 2, furthercomprising identifying a face of the person based on the informationreceived from the one or more optical sensors, wherein determiningwhether the person is further than the threshold distance from thedisplay device or closer than the threshold distance from the displaydevice is based on a size of the face of the person.
 4. The method ofclaim 1, wherein the person is determined to be further than thethreshold distance from the display device, the method furthercomprising: while the first user interface is displayed on the displaydevice, detecting movement of the person to a distance closer than thethreshold distance from the display device; and in response to detectingthe movement of the person to the distance closer than the thresholddistance, providing the second user interface for display on the displaydevice.
 5. The method of claim 1, wherein the person is determined to becloser than the threshold distance from the display device, the methodfurther comprising: while the second user interface is displayed on thedisplay device, detecting movement of the person to a distance furtherthan the threshold distance from the display device; and in response todetecting the movement of the person to the distance further than thethreshold distance, providing the first user interface for display onthe display device.
 6. The method of claim 1, wherein: the person isfurther than the threshold distance from the display device, and thefirst user interface is provided for display on the display device, thefirst user interface including a first user interface element and asecond user interface element, the method further comprising: while thefirst user interface is displayed on the display device: determining agaze location in the first user interface, wherein the gaze locationcomprises a location in the first user interface to which a gaze of theperson is directed; detecting movement of the person to a distancecloser than the threshold distance from the display device; and inresponse to detecting the movement of the person, providing the seconduser interface for display on the display device, the second userinterface including information based on the gaze location.
 7. Themethod of claim 6, further comprising: determining whether the gazelocation coincides with the first user interface element or the seconduser interface element in the first user interface, wherein in responseto detecting the movement of the person to the distance closer than thethreshold distance from the display device: in accordance with adetermination that the gaze location coincides with the first userinterface element, providing the second user interface for display onthe display device, the second user interface including informationassociated with the first user interface element, and in accordance witha determination that the gaze location coincides with the second userinterface element, providing the second user interface for display onthe display device, the second user interface including informationassociated with the second user interface element.
 8. The method ofclaim 1, wherein: the person is closer than the threshold distance fromthe display device, and the person is logged into one or more areas onthe electronic device, the method further comprising: detecting movementof the person to a distance further than the threshold distance from thedisplay device; and in response to detecting the movement of the personto the distance further than the threshold distance from the displaydevice, logging out of the one or more areas on the electronic device.9. The method of claim 1, wherein the first user interface includes auser interface element at a first size, and the second user interfaceincludes the user interface element at a second size, smaller than thefirst size.
 10. The method of claim 1, wherein the first user interfaceincludes summary information about a first quantity, and the second userinterface includes detailed information about the first quantity. 11.The method of claim 1, wherein the first user interface includes a firstset of user interface elements, and the second user interface includesthe first set of user interface elements and a second set of userinterface elements.
 12. An electronic device, comprising: one or moreprocessors; memory; and one or more programs, wherein the one or moreprograms are stored in the memory and configured to be executed by theone or more processors, the one or more programs including instructionsfor: determining whether a person is further than a threshold distancefrom a display device or closer than the threshold distance from thedisplay device; in accordance with a determination that the person isfurther than the threshold distance from the display device, providing afirst user interface for display on the display device; and inaccordance with a determination that the person is closer than thethreshold distance from the display device, providing a second userinterface for display on the display device, wherein the second userinterface is different than the first user interface.
 13. Anon-transitory computer readable storage medium storing one or moreprograms, the one or more programs comprising instructions, which whenexecuted by an electronic device with one or more processors and memory,cause the electronic device to perform a method comprising: determiningwhether a person is further than a threshold distance from a displaydevice or closer than the threshold distance from the display device; inaccordance with a determination that the person is further than thethreshold distance from the display device, providing a first userinterface for display on the display device; and in accordance with adetermination that the person is closer than the threshold distance fromthe display device, providing a second user interface for display on thedisplay device, wherein the second user interface is different than thefirst user interface.